Data tier ergonomics

[jpountz]

The formalization of data tiers (#60994) enables us to ship with defaults that are better suited for the expected usage pattern.

Content indices

If we can expect content indices to be more actively searched than indexed, we could make the following changes, which would trade indexing speed for search speed or

• Number of segments per tier set to 5 (default: 10)?
• Maximum percentage of deletes set to 20% (default: 33%)?
• Enable merge-on-refresh (default: disabled)?
• Refresh interval set to 1s (default: disabled on inactive shards, 1s on active shards)?

Hot indices

Our defaults tend to be geared towards hot data, is there any setting that could have a more sensible value on hot data?

Warm indices

Warm, cold and frozen indices are expected to be allocated to dense storage. Memory needs to be wisely used.

• Disable query cache (Disable query cache for density-optimized use-cases? #59846)
• Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)

Cold indices

• Disable query cache (Disable query cache for density-optimized use-cases? #59846)
• Automatically leverage FrozenEngine (Should we automatically use the FrozenEngine on cold and frozen tiers? #71002)
• Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)

Frozen indices

• Disable query cache (Disable query cache for density-optimized use-cases? #59846)
• Automatically leverage FrozenEngine (Should we automatically use the FrozenEngine on cold and frozen tiers? #71002)
• Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[javanna]

There are no concrete plans to work on this meta issue and its sub-tasks. Closing.