Enable Stack Features (X-Pack) and trial by default

[antoineco]

We're now "secure by default" with X-Pack and paid features enabled by default 🔐 (30-day trial)

If this gets merged, the x-pack branch will become the new master in a second step.

Closes #163
Closes #378

[deviantony]

I did a quick test by rebuilding the stack entirely through:

docker-compose down -v
docker-compose build
docker-compose up

But it seems that there are a lot of authentication issues between the services now:

kibana_1         | {"type":"log","@timestamp":"2019-04-26T21:05:19Z","tags":["warning","task_manager"],"pid":1,"message":"PollError [security_exception] failed to authenticate user [kibana], with { header={ WWW-Authenticate=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } }"}

elasticsearch_1  | [2019-04-26T21:05:19,775][INFO ][o.e.x.s.a.AuthenticationService] [yecAKML] Authentication of [kibana] was terminated by realm [reserved] - failed to authenticate user [kibana]

elasticsearch_1  | [2019-04-26T21:05:20,684][INFO ][o.e.x.s.a.AuthenticationService] [yecAKML] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

logstash_1       | [2019-04-26T21:05:20,686][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http:https://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http:https://elasticsearch:9200/'"}

I think it's related to the fact that you changed the default elastic username and removed ELASTIC_PASSWORD from the compose file. I can try to investigate a bit later.

[antoineco]

I think it's related to the fact that you changed the default elastic username and removed ELASTIC_PASSWORD from the compose file. I can try to investigate a bit later.

Yes it's by design (see new README section in "Initial setup"). We kind of abused this bootstrap password on the X-Pack branch, whereas the documentation recommends using the built-in users with less privileges instead. Just trying to apply best practices and making user aware of security. I can change it back and use the super admin everywhere.

[deviantony]

@antoineco oh I see, you got a point with security awareness.

Argh, another battle security VS user experience :-)

My opinion is to use the super-admin everywhere in order to make the experience using this stack pretty much a "clone and up". We're not really providing a production ready stack, the main goal is still to be able to quickly give a try to the Elastic stack hassle free.

tbh, this is kinda what I was expected when I tested this branch, just clone and up.

WDYT about default super-admin and a note about security in the README?

[antoineco]

Sounds good! I'll do that.

[antoineco]

@deviantony Done here and here.
@mw-jko your review is also welcome :)

[deviantony]

Overall LGTM.

Quickly tested, seems to be working fine.

[deviantony]

We can remove the vagrant branch ref, I don't think it is worth maintaining anymore.

[antoineco]

Done

[deviantony]

Worth adding a quick note here saying that this can be easily disabled so that we don't frighten people that don't want to use paid features.

[antoineco]

Done

[deviantony]

Is this outdated?

[antoineco]

I just rephrased it and changed the link to point to the official documentation, which didn't exist back then.

[deviantony]

Oh sorry, missed the link, LGTM

[deviantony]

LGTM

[deviantony]

Good work ! 👍

Not sure why the travis build fails though. I'll let you merge this one.

[antoineco]

Looks like Travis is sometimes slower and Logstash is not ready when it should be. I'll bump the delays a little bit if it fails this round again.

Thanks for the review! 🙌

[j-koehler]

@mw-jko your review is also welcome :)

Sorry, I was on vacation 🙈

edit:

LGTM but did not try it by myself.