Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New feature gelf "raw" message processing #47915

Open
131 opened this issue Jun 5, 2024 · 2 comments
Open

New feature gelf "raw" message processing #47915

131 opened this issue Jun 5, 2024 · 2 comments
Labels
area/logging kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny

Comments

@131
Copy link

131 commented Jun 5, 2024

Description

Today,the gelf driver take a line and use it as the "short" version of a message, adding all metadata in a surrounding JSON armor.

I suggest a new gelf-raw-json option to parse every incoming log line as a JSON payload, and merge all drivers metadata in it.
That will dramaticaly increase the gelf driver capability yet, in a safe design.

This is simple enough for me to implement, here is the related PR

@131 131 added kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny status/0-triage labels Jun 5, 2024
@cpuguy83
Copy link
Member

cpuguy83 commented Jun 5, 2024

to parse every incoming log line as a JSON payload, and merge all drivers metadata in it.

What does this mean? Do you have an example?

@131
Copy link
Author

131 commented Jun 5, 2024

What does this mean? Do you have an example?

Let say my application output (on stderr) native JSON payload
{"message": "this is something", "source_file": "/path/to/my/exe", "severity": "WHATEVER"}

Currently, using the GELF driver, all this line will end up in a super structure

{
     "_container_id":   info.ContainerID,
    "_container_name": info.Name(),
    "_image_id":       info.ContainerImageID,
    "_image_name":     info.ContainerImageName,
    "_command":        info.Command(),
    "_tag":            tag,
    "_created":        info.ContainerCreated,
    "message" : "{\"message\":\"this is something\",\"source_file\":\"/path/to/my/exe\",\"severity\":\"WHATEVER\"}"

}

All metadata will be lost, and post processing will be required.

With gelf-json-raw = true, gelf will receive,

{
     "message": "this is something",
     "source_file": "/path/to/my/exe",
     "severity": "WHATEVER",

     "_container_id":   info.ContainerID,
    "_container_name": info.Name(),
    "_image_id":       info.ContainerImageID,
    "_image_name":     info.ContainerImageName,
    "_command":        info.Command(),
    "_tag":            tag,
    "_created":        info.ContainerCreated,
}

All metadata are preserved and usable by gralog.

Please note that all native driver field will take precedence on untrusted payload.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/logging kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny
Projects
None yet
Development

No branches or pull requests

3 participants