entire .lz4 files must be read before processing

[jtmoon79]

Summary

The entire .lz4 file is uncompressed to get it's uncompressed size before processing.

Current behavior

Crate lz4_flex does not provide an API for getting the .lz4 file uncompressed size. See PSeitz/lz4_flex#159

So the entire file is uncompressed to get the uncompressed file size.

Suggested behavior

One or both

• Submit a PR to lz4_flex.
• refactor blockreader (and all readers) to not need a file size (difficult)

Also see

Similar to #300 regarding .bz2
Meta-Issue #182

[jtmoon79]

Follow on from #291

[jtmoon79]

By default, .lz4 files compressed by lz4c do not include a content_size.

$ lz4c --help
*** LZ4 command line interface 64-bits v1.9.3, by Yann Collet ***
Usage :
      lz4c [arg] [input] [output]

...
--content-size : compressed frame includes original size (default:not present)

So even if I could access content_size in the FrameHeader, it would most often not be valid. So the current implementation of first reading the entire file would still be required.