proposal: Add a "file proxy format"

[bep]

This is related to #5749 -- but I think that concept has a wider potential even if the title of this issue may seem a little out there (I just made it up quickly), so I quickly jot down some thoughts here.

version https://git-lfs.github.com/spec/v1
oid sha256:e6df77690697794deacb9d6963e574044448bef923f7d688aab1d856c54388a3
size 4074082

The above is an example of a Git LFS placeholder for an image. It doesn't say where to find the image, but I guess it assumes to be in the current "Git LFS" repo.

We will add remote support for resource.Get so you can do {{ resource.Get "https://bep.is/sunset.img }} etc. which is super cool, but has some limitations:

• Without additional information we would have to download the file (or an ETag) on every build to check if it has changed.

• It would not work in page bundles.

  If we could create a bundle:

my-bundle
├── index.md
└── sunset.jpg

And sunset.jpg would contain something ala:

version https://gohugo.io/resources/v1
source: https://bep.is/sunset.jpg
sha256:e6df77690697794deacb9d6963e574044448bef923f7d688aab1d856c54388a3
size 4074082

The above would probably need some help from some utility (like hugo fetch image) because it would be too hard to write by hand. But it has some benefits:

1. It is very generic and could be used for anything.
2. The SHA256 would allow for very fast cache evaluation.
3. It would take almost no place on disk, and could be copied freely if you need to use the image etc. in another bundle.
4. We could add metadata to it (title, params etc.)

[moorereason]

Interesting idea. Random comments:

Without additional information we would have to download the file (or an ETag) on every build to check if it has changed.

If the hosting server allows it, we could use HTTP HEAD to retrieve headers without the body.

Are you proposing to cache these resources elsewhere in the project and commit the cache to git? I'm concerned about reproducible builds.

I'd tweak the format just a tad to add a hash keyword. Here's an ETag MD5 example:

version https://gohugo.io/resources/v1
source https://bepsays.com/assets/img/hugotree.jpg
hash md5:b426aee4ccaa45c03baa2b6c301cf371
size 333526

[bep]

Are you proposing to cache these resources elsewhere in the project and commit the cache to git? I'm concerned about reproducible builds.

No, I'm proposing to use the existing file cache in Hugo, which can be used however you will:

• Commit to Git
• Configure to use Netlify's (or Circle CI etc.) file cache -- this is what I primarily have in mind
• Use a /tmp folder for cache

Note that with remote resources there will be plenty of error scenarios, but the "reproducible builds" would be bulletproof when it works (assuming you trust SHA256).