Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add option for Append Processor to skip/allow empty values #105718

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add option for Append Processor to skip/allow empty values #105718

wants to merge 2 commits into from
+204 −32

Conversation

limotova
Copy link
Contributor

If one is using templates to generate values, this option can be helpful to avoid adding values that resolve into empty strings

Fixes #104813

@limotova
Add option for Append Processor to skip/allow empty values
5cbbd5e 
If one is using templates to generate values, this option can be helpful
to avoid adding values that resolve into empty strings

Fixes elastic#104813
@github-actions GitHub Actions
Copy link

Documentation preview:

@elasticsearchmachine elasticsearchmachine added v8.14.0 needs:triage Requires assignment of a team area label external-contributor Pull request authored by a developer outside the Elasticsearch team labels Feb 22, 2024
@limotova limotova mentioned this pull request Feb 22, 2024
Open
@tlrx tlrx added the :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP label Feb 22, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Feb 22, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Feb 22, 2024
@joegallo joegallo self-requested a review March 8, 2024 18:18
joegallo
joegallo requested changes Mar 8, 2024
View reviewed changes
Copy link
Contributor

@joegallo joegallo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to have this mirror the set processor's ignore_empty_value option, rather than creating a new option name with no precedent -- it's functionally the same as that option, right (albeit with the opposite sense)?

@limotova limotova requested a review from joegallo March 9, 2024 04:40
@joegallo joegallo self-assigned this Mar 29, 2024
@romulets
Copy link
Member

This feature is actually pretty useful. I'm my use-case, I'm appending a lot of fields. To keep checking if they are not null on context is pretty repetitive. Right now we have a painless script that removes all empty values. But would be awesome to have this out of the box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joegallo joegallo Awaiting requested review from joegallo

Requested changes must be addressed to merge this pull request.

Assignees

@joegallo joegallo

Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Data Management Meta label for data/management team v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

FR: Allow Append processor to ignore empty values
6 participants
@limotova @elasticsearchmachine @romulets @joegallo @tlrx @mark-vieira