Move parent-join and nested in a plugin/module

[jimczi]

Listing these features as primitives in our mapping can be misleading for users. The norm when using a search engine like ES should be to denorrmalize the data upfront. It's too easy to start with a nested model or a complex parent-join and realize the limitations of the query/aggs later. It is also not sure that the nested model saves a lot these days with all the improvements in Lucene around compressions and sparse values support.

It's also hard to control the scope of these features if they live in core. We have some features where nested "should" be supported but not actually tested because we never thought that it would be used in this context.

I propose that we move the parent-join to an official plugin for now. It can be used in specific contexts with a pre-defined scope but we don't provide it by default. The options that we have in our default deployment should be investigated first. Global solutions on top of ES shouldn't rely on this feature either or require to install the plugin first.

For nested, I understand that it will not be a popular opinion but it would be nice to remove the support from our default distribution too. We can first try to move it out of core, lots of the complexity in our mappings, queries and aggregations come from the support of this feature so it would be nice to isolate it in a module. If it's feasible it will be easier to remove it entirely in a future release or to develop alternatives that don't require to change the entire query/aggs model to work.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[javanna]

We have no concrete plans to work on this issue. Closing.