handling gzip encoded http responses?

[huyng]

A lot content on the web is hosted using gzip encoding.

For example the following command produces a lot of binary data output since github responds with gzip encoding by default.

megfile head https://raw.githubusercontent.com/huyng/table-bench/main/table.csv

To see the raw text, you have to pipe it through gunzip like so:

megfile head https://raw.githubusercontent.com/huyng/table-bench/main/table.csv | gunzip

Do you plan on supporting this gzip streaming in your library? Thank you for creating this amazing tool.

[LoveEatCandy]

I will support decoding gzip content over http, is that enough for you?
I think http is a special situation that i should decode the gzip content like browser, but .gz files should be decoded by user.

[huyng]

yes I think the behavior you outline is the one I had in mind as well. Mainly: if the server sends back an http header of Content-Encoding: gzip or Content-Encoding: deflate then the library will take care of gunzipping the content for the user.

[LoveEatCandy]

Header Content-Encoding has been supported in megfile>=3.0.5. Thank you for your issue, it's very important for us.

[huyng]

Excellent it works! Thank you for adding this functionality.