Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support more field types for ignore_malformed #99400

Open
felixbarny opened this issue Sep 11, 2023 · 4 comments
Open

Support more field types for ignore_malformed #99400

felixbarny opened this issue Sep 11, 2023 · 4 comments
Labels
>enhancement :StorageEngine/Logs You know, for Logs :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine

Comments

@felixbarny
Copy link
Member

Based on testing

There are a couple of field types that don't support ignore_malformed, yet:

  • ip_range
  • integer_range
  • date_range
  • object
  • flattened
  • nested
@felixbarny felixbarny added the :Search Foundations/Mapping Index mappings, including merging and defining field types label Sep 11, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Sep 11, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@martijnvg martijnvg added :StorageEngine/Mapping The storage related side of mappings and removed :Search Foundations/Mapping Index mappings, including merging and defining field types labels May 31, 2024
@elasticsearchmachine elasticsearchmachine added Team:StorageEngine and removed Team:Search Meta label for search team labels May 31, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@salvatore-campagna
Copy link
Contributor

As we discussed in in the Logs & Metrics - weekly sync of 27 June 2024 we will not prioritize fixing this for logs since this is already an issue with standard indexing and we do not expect mappings with such field types to use ignore_malformed. That would result in a mapping error no matter the index mode.

@felixbarny
Copy link
Member Author

This is more of a feature request that's orthogonal to LogsDB. It would help that the index-level ignore_malformed setting is more effective in avoiding rejecting a whole document when a single value is malformed. I agree that this isn't required to make LogsDB GA, but it would help in our effort to "accept all logs".

@lkts lkts added the :StorageEngine/Logs You know, for Logs label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :StorageEngine/Logs You know, for Logs :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine
Projects
None yet
Development

No branches or pull requests

6 participants