Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flow.log: Async_file_logger: Add gzip-on-the-fly; gzip chunks; etc. #79

Open
ygoldfeld opened this issue Mar 10, 2024 · 0 comments
Open
Labels
enhancement New feature or request from-akamai-pre-open Issue origin is Akamai, before opening source

Comments

@ygoldfeld
Copy link
Contributor

ygoldfeld commented Mar 10, 2024

Filed by @ygoldfeld pre-open-source:

(Various services) can output logs directly in gzip form. (gzip? And/or other compression formats too?) Async_file_logger, at least, should support the same. 

The naive thing, which is to output one gzip stream, should be easy with Boost.stream which has a gzip filter (EDIT: but consider other possibilities). The ostream writer in flow.log could be easily adapted to use it.

(colleague) also tells me that (some services) don't output one gzip stream (or maybe it is optional). (They) output little gzip streams (one per message? one per certain # of lines or messages? check), each one valid and then followed by some sentinel or something (check it). The idea is that if the file is corrupted, the non-corrupted parts can still be decoded fine.

So do that too.

All details TBD.

But the general feature is clearly a good one.

@ygoldfeld ygoldfeld added enhancement New feature or request from-akamai-pre-open Issue origin is Akamai, before opening source labels Mar 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request from-akamai-pre-open Issue origin is Akamai, before opening source
Projects
None yet
Development

No branches or pull requests

1 participant