ReadonlyREST is an GPLv3 open source project. Its ongoing development can only made possible thanks to the support of its backers:
- @nmaisonneuve
- @Id57
- Joseph Bull
If you care this project keeps on existing, read up the ReadonlyREST Patreon campaign.
Expose the high performance HTTP server embedded in Elasticsearch directly to the public, safely blocking any attempt to delete or modify your data.
In other words... no more proxies! Yay Ponies!
Download the binary release for your Elasticsearch version from the official website.
Append either of these snippets to conf/elasticsearch.yml
USE CASE: Secure public searchbox from ransomware
readonlyrest:
enable: true
template_rules:
- name: "Accept all requests from localhost"
type: allow
hosts: [127.0.0.1]
- name: "::PUBLIC SEARCHBOX::"
type: allow
indices: ["public"]
actions: ["indices:data/read/*"]
Remember to enable SSL whenever you use HTTP basic auth or API keys so your credentials can't be stolen.
http.type: ssl_netty4
readonlyrest:
enable: true
ssl:
enable: true
keystore_file: "/elasticsearch/plugins/readonlyrest/keystore.jks"
keystore_pass: readonlyrest
key_pass: readonlyrest
readonlyrest:
enable: true
response_if_req_forbidden: Sorry, your request is forbidden.
template_rules:
- name: Accept all requests from localhost
type: allow
hosts: [127.0.0.1]
- name: Just certain indices, and read only
type: allow
actions: ["indices:data/read/*"]
indices: ["product_catalogue-*"] # index aliases are taken in account!
readonlyrest:
enable: true
ssl:
enable: true
keystore_file: "/elasticsearch/plugins/readonlyrest/keystore.jks"
keystore_pass: readonlyrest
key_pass: readonlyrest
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: "::LOGSTASH::"
# auth_key is good for testing, but replace it with `auth_key_sha1`!
auth_key: logstash:logstash
type: allow
actions: ["cluster:monitor/main","indices:admin/types/exists","indices:data/read/*","indices:data/write/*","indices:admin/template/*","indices:admin/create"]
indices: ["logstash-*"]
# We trust Kibana's server side process, full access granted via HTTP authentication
- name: "::KIBANA-SRV::"
# auth_key is good for testing, but replace it with `auth_key_sha256`!
auth_key: kibana:kibana
verbosity: error # don't log successful request
type: allow
# Using "Basic HTTP Auth" from browsers, can RW Kibana settings, RO on logstash indices from 2017 .
- name: "::RW DEVELOPER::"
auth_key: rw:dev
type: allow
kibana_access: rw
indices: [".kibana", ".kibana-devnull", "logstash-2017*"]
# Same as above, but cannot change dashboards, visualizations or settings in Kibana
- name: "::RO DEVELOPER::"
auth_key: ro:dev
type: allow
kibana_access: ro
indices: [".kibana", ".kibana-devnull", "logstash-2017*"]
Now activate authentication in Kibana server: let the Kibana daemon connect to ElasticSearch in privileged mode.
- edit the kibana configuration file:
kibana.yml
and add the following:
elasticsearch.username: "kibana"
elasticsearch.password: "kibana"
This is secure because the users connecting from their browsers will be asked to login separately anyways.
Now activate authenticatoin in Logstash: (follow the docs, it's very similar to Kibana!)
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: Accept requests from users in group team1 on index1
type: allow
groups: ["team1"]
indices: ["index1"]
- name: Accept requests from users in group team2 on index2
type: allow
groups: ["team2"]
indices: ["index2"]
- name: Accept requests from users in groups team1 or team2 on index3
type: allow
groups: ["team1", "team2"]
indices: ["index3"]
users:
- username: alice
auth_key: alice:p455phrase
groups: ["team1"]
- username: bob
auth_key: bob:s3cr37
groups: ["team2", "team4"]
- username: claire
auth_key_sha1: 2bc37a406bd743e2b7a4cb33efc0c52bc2cb03f0 #claire:p455key
groups: ["team1", "team5"]
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: Accept requests from users in group team1 on index1
type: allow
ldap_auth:
name: "ldap1" # ldap name from below 'ldaps' section
groups: ["g1", "g2"] # group within 'ou=Groups,dc=example,dc=com'
indices: ["index1"]
- name: Accept requests from users in group team2 on index2
type: allow
ldap_auth:
- name: "ldap2"
groups: ["g3"]
cache_ttl_in_sec: 60
indices: ["index2"]
ldaps:
- name: ldap1
host: "ldap1.example.com"
port: 389 # optional, default 389
ssl_enabled: false # optional, default true
ssl_trust_all_certs: true # optional, default false
bind_dn: "cn=admin,dc=example,dc=com" # optional, skip for anonymous bind
bind_password: "password" # optional, skip for anonymous bind
search_user_base_DN: "ou=People,dc=example,dc=com"
user_id_attribute: "uid" # optional, default "uid"
search_groups_base_DN: "ou=Groups,dc=example,dc=com"
unique_member_attribute: "uniqueMember" # optional, default "uniqueMember"
connection_pool_size: 10 # optional, default 30
connection_timeout_in_sec: 10 # optional, default 1
request_timeout_in_sec: 10 # optional, default 1
cache_ttl_in_sec: 60 # optional, default 0 - cache disabled
- name: ldap2
host: "ldap2.example2.com"
port: 636
search_user_base_DN: "ou=People,dc=example2,dc=com"
search_groups_base_DN: "ou=Groups,dc=example2,dc=com"
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: Accept requests from users in group team1 on index1
type: allow
ldap_authentication: "ldap1"
ldap_authorization:
name: "ldap1" # ldap name from 'ldaps' section
groups: ["g1", "g2"] # group within 'ou=Groups,dc=example,dc=com'
indices: ["index1"]
- name: Accept requests from users in group team2 on index2
type: allow
ldap_authentication:
name: "ldap2"
cache_ttl_in_sec: 60
ldap_authorization:
name: "ldap2"
groups: ["g3"]
cache_ttl_in_sec: 60
indices: ["index2"]
ldaps:
- name: ldap1
host: "ldap1.example.com"
port: 389 # default 389
ssl_enabled: false # default true
ssl_trust_all_certs: true # default false
bind_dn: "cn=admin,dc=example,dc=com" # skip for anonymous bind
bind_password: "password" # skip for anonymous bind
search_user_base_DN: "ou=People,dc=example,dc=com"
user_id_attribute: "uid" # default "uid"
search_groups_base_DN: "ou=Groups,dc=example,dc=com"
unique_member_attribute: "uniqueMember" # default "uniqueMember"
connection_pool_size: 10 # default 30
connection_timeout_in_sec: 10 # default 1
request_timeout_in_sec: 10 # default 1
cache_ttl_in_sec: 60 # default 0 - cache disabled
- name: ldap2
host: "ldap2.example2.com"
port: 636
search_user_base_DN: "ou=People,dc=example2,dc=com"
search_groups_base_DN: "ou=Groups,dc=example2,dc=com"
LDAP configuration requirements:
- user from
search_user_base_DN
should haveuid
attribute (can be overwritten usinguser_id_attribute
) - groups from
search_groups_base_DN
should haveuniqueMember
attribute (can be overwritten usingunique_member_attribute
)
(An example OpenLDAP configuration file can be found in our tests: /src/test/resources/test_example.ldif)
Caching can be configured per LDAP client (see ldap1
) or per rule (see Accept requests from users in group team2 on index2
rule)
ReadonlyREST will forward the received Authorization
header to a website of choice and evaluate the returned HTTP status code to verify the provided credentials.
This is useful if you already have a web server with all the credentials configured and the credentials are passed over the Authorization
header.
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: "::Tweets::"
type: allow
methods: GET
indices: ["twitter"]
external_authentication: "ext1"
- name: "::Facebook posts::"
type: allow
methods: GET
indices: ["facebook"]
external_authentication:
service: "ext2"
cache_ttl_in_sec: 60
external_authentication_service_configs:
- name: "ext1"
authentication_endpoint: "https://external-website1:8080/auth1"
success_status_code: 200
cache_ttl_in_sec: 60
- name: "ext2"
authentication_endpoint: "https://external-website2:8080/auth2"
success_status_code: 204
cache_ttl_in_sec: 60
To define an external authentication service the user should specify:
name
for service (then this name is used as id inservice
attribute ofexternal_authentication
rule)authentication_endpoint
(GET request)success_status_code
- authentication response success status code
Cache can be defined at the service level or/and at the rule level. In the example, both are shown, but you might opt for setting up either.
This external authorization connector makes it possible to resolve to what groups a users belong, using an external JSON or XML service.
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
template_rules:
- name: "::Tweets::"
type: allow
methods: GET
indices: ["twitter"]
proxy_auth:
proxy_auth_config: "proxy1"
users: ["*"]
groups_provider_authorization:
user_groups_provider: "GroupsService"
groups: ["group3"]
- name: "::Facebook posts::"
type: allow
methods: GET
indices: ["facebook"]
proxy_auth:
proxy_auth_config: "proxy1"
users: ["*"]
groups_provider_authorization:
user_groups_provider: "GroupsService"
groups: ["group1"]
cache_ttl_in_sec: 60
proxy_auth_configs:
- name: "proxy1"
user_id_header: "X-Auth-Token" # default X-Forwarded-User
user_groups_providers:
- name: GroupsService
groups_endpoint: "https://localhost:8080/groups"
auth_token_name: "token"
auth_token_passed_as: QUERY_PARAM # HEADER OR QUERY_PARAM
response_groups_json_path: "$..groups[?(@.name)].name" # see: https://github.com/json-path/JsonPath
cache_ttl_in_sec: 60
In example above, a user is authenticated by reverse proxy and then external service is asked for groups for that user.
If groups returned by the service contain any group declared in groups
list, user is authorized and rule matches.
To define user groups provider you should specify:
name
for service (then this name is used as id inuser_groups_provider
attribute ofgroups_provider_authorization
rule)groups_endpoint
- service with groups endpoint (GET request)auth_token_name
- user identifier will be passed with this nameauth_token_passed_as
- user identifier can be send using HEADER or QUERY_PARAMresponse_groups_json_path
- response can be unrestricted, but you have to specify JSON Path for groups name list (see example in tests)
As usual, the cache behaviour can be defined at service level or/and at rule level.
This authentication is based on OAuth. The goal is to authenticate a user with an access token.
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
oauth_enabled: true
cookieSecret: secret
cookieName: myCookie
tokenClientId: demo
tokenSecret: publickey
In the example above, the user is authenticated with a JWT token (https://jwt.io/) set in a cookie called 'myCookie' Only RSA-256 (for now) encryption is supported as JWT signature.
cookieSecret
is the secret used to encrypt the cookie (with Jiron)cookieName
is the name of the CookietokenClientId
is the name used to get the groups of the user in the token, under the claim "resource_access"tokenSecret
the public key (in case of RSA-256) used to sign and verify the token
In order to make this use case work, you'll need to install https://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html in your JAVA_HOME
For other use cases and finer access control have a look at the official documentation to see the full list of supported rules
Before going to production, read this.
When you want to restrict access to certain indices, in order to prevent the user from overriding the index which has been specified in the URL, add this setting to the config.yml file:
rest.action.multi.allow_explicit_index: false
The default value is true, but when set to false, Elasticsearch will reject requests that have an explicit index specified in the request body.
Plain text auth_key
is is great for testing, but remember to replace it with auth_key_sha256
!
Other security plugins are replacing the high performance, Netty based, embedded REST API of Elasticsearch with Tomcat, Jetty or other cumbersome XML based JEE madness.
This plugin instead is just a lightweight pure-Java filtering layer. Even the SSL layer is provided as an extra Netty transport handler.
Some suggest to spin up a new HTTP proxy (Varnish, NGNix, HAProxy) between ES and clients to filter out malicious access with regular expressions on HTTP methods and paths. This is a bad idea for two reasons:
- You're introducing more complexity in your architecture.
- Reasoning about security at HTTP level is risky, flaky and less granular than controlling access at the internal ElasticSearch protocol level.
The only clean way to do the access control is AFTER ElasticSearch has parsed the queries.
Just set a few rules with this plugin and confidently open it up to the external world.
Build your ACL from simple building blocks (rules) i.e.:
hosts
a list of origin IP addresses or subnets
api_keys
a list of api keys passed in via headerX-Api-Key
methods
a list of HTTP methodsaccept_x-forwarded-for_header
interpret theX-Forwarded-For
header as origin host (useful for AWS ELB and other reverse proxies)auth_key_sha1
HTTP Basic auth (credentials stored as hashed strings).uri_re
Match the URI path as a regex.
indices
indices (aliases and wildcards work)actions
list of ES actions (e.g. "cluster:" , "indices:data/write/", "indices:data/read*")
kibana_access
captures the read-only, read-only + new visualizations/dashboards, read-write use cases of Kibana.
This project was incepted in this StackOverflow thread.
Thanks Ivan Brusic for publishing this guide