WARC file support?

[anarcat]

it would be nice if this project would save files in WARC format. This allows collecting multiple page elements (JS, CSS, images, etc) in a single file. It's the standard archival format for web pages that is used by archive.org and many other programs. wget has support for it, for example.

This allows more faithful playback of pages than raw HTML because it includes HTTP headers which are sometimes capital in proper playback (if only, for example, for Content-type headers).

[RadhiFadlillah]

This issue is related with #108. So, archival is supported now, but not in warc format. This is done because IMHO warc is not really suitable for Shiori.

Correct me if I'm wrong but as far as I understand warc is simply a file that contains all of web page resources in text format. Since it's text based, it's not really easy to parse, and a bit slow to access a specific resource. I also not sure I could make the parser thread-safe.

Instead I just use bbolt which is key-value database. Thanks to this, read and write is easy and thread-safe. Not to mention the bbolt API is now fixed, so I believe it's safe enough to use for archival.

With that said, I believe the archival has been implemented albeit not in warc format, so I will close this issue. Feel free to reopen if needed.

[neezer]

If not using WARC proper, why not just use a file dump to a folder with the HTML, CSS, images, etc.? I'd like to make changes to the archive file because Shiori isn't correctly handling some things yet (#230, for example), but at the moment the archive exists as an opaque file—I don't have much of an idea about how to go about editing it manually.

I don't need wget replay or interoperability with existing WARC-processing tools like @anarcat, so I'm struggling to see the end-user value of a single-file archive: what does it deliver that a folder of assets wouldn't?