Firebase deploy should not reupload unchanged files

[hickford]

My site is 0.2 MB of html and 9.8 MB of images. I have a slow internet connection. Whenever I change the html, I suffer firebase deploy reuploading all the images—even though they are unchanged.

Good old Divshot was careful to compare sha256 sums so it could skip reuploading unchanged files. This made it usable on a slow connection. You can read a description of its algorithm at https://github.com/divshot/divshot-push/blob/master/lib/sync-tree.js

Whereas Firebase simply creates a tar ball of the entire project firebase-upload-*.tar.gz https://github.com/firebase/firebase-tools/blob/master/lib/prepareUpload.js , including all the unchanged files.

Please think how to conserve bandwidth, for those of us not blessed with high speed internet.

[hickford]

See also similar feature request for Surge.sh sintaxi/surge#119

[mbleigh]

Yep, this is on the roadmap and we know it's strongly desired. Can't give
you a timeline, but we're well aware.

On Tue, May 17, 2016, 9:49 AM Mirth Hickford [email protected]
wrote:

See also similar feature request for Surge.sh sintaxi/surge#119
sintaxi/surge#119

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#133 (comment)

[hickford]

Hi Mike. Great, look forward to it.

[dht]

Also looking forward. +1

[nicogarcia]

+1

[TheoMer]

+1

[seanmavley]

So is it fixed, @mbleigh ?

[mbleigh]

No. We're still interested in tackling this, but can't give a timeline on when it will happen.

…

On Wed, Jan 4, 2017 at 12:33 PM Nkansah Rexford ***@***.***> wrote: So is it fixed, @mbleigh <https://github.com/mbleigh> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAD_jciGq4eqtEYVL1lUXp3tzIXl9xrks5rPAIdgaJpZM4Igep8> .

[jjmpsp]

It's my first time using firebase hosting and I have to say I was surprised to discover firebase uploads the entire public folder upon calling the firebase deploy command. I'm so used to using tools which calculates file differences (Git, Unity collaborate, etc.) so I thought the firebase deploy command was hanging (or failing) and spent a good 30 minutes searching online for a solution to the 'problem'. Turns out the process was working fine after all, but just took its time over my slow internet connection.

Just as a FYI, if you want to track the upload progress, simply open an Activity Manager application from your system OS and monitor the 'node' process for upload activity:

[kaankucukx]

nothing yet ?

[mbleigh]

No, nothing yet. There is substantial infrastructure change that has to happen to support this and while we're still actively interested in making it happen, we don't have a timeline to share.

…

On Sun, Apr 9, 2017, 11:34 AM kaankucukx ***@***.***> wrote: nothing yet ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAD_vtWyxv3X7STl9h4xDd-_TkI0XR8ks5ruSS_gaJpZM4Igep8> .

[coreybutler]

@mbleigh - while you're all thinking about it, I wanted to mention my use case for consideration as well. I have an "append-only" site where new static content is created on a fairly regular basis. I always know what's new... there's never any diffing required in my particular use case. I'd be thrilled to see the ability to tack on new content, even if I had to make the zip/tarball myself, i.e. something like firebase deploy --append ./my.zip. Just food for thought.

Also, if there's anything I can do to help, let me know. I'm pretty handy with Node and static web hosting :-)

[Ovi]

+1

[literalpie]

Is this still in the works? Any updates?

[talas9]

This feature is really important, any small change to any function requires redeployment of all other functions, this means all the functions on the cloud will re download the npm packages on first-run of each untouched function.

This should be a priority, please share any ETA you have on this.

[laurenzlong]

@talas9 This issue is for Firebase Hosting, you're actually talking about Functions. See https://firebase.google.com/docs/cli/#partial_deploys for instructions on how to deploy single or groups of functions instead of all of them.

[talas9]

@laurenzlong sorry for that, so I just have a question: does deploying selected functions (partial deployment) will keep the ones not selected for deployment and already running in the cloud? or it will delete them?

[laurenzlong]

@talas9 No problem! It keeps them running as they are.

[maxcorbeau]

@laurenzlong @mbleigh : hi guys, any ETA on this?

[pedrommuller]

any update on this feature?

[samtstern]

This is the 5th most "reacted" open issue on Firebase, so I am really happy to be able to close it!

https://firebase.googleblog.com/2018/08/one-project-multiple-sites-plus-boost.html

If you updated the Firebase CLI recently, you might have noticed that your uploads got a bit faster. You may have also noticed a new .firebase folder in your project. That's because we rolled up a new deployment system that we call Delta Uploads.

This new system only processes new, modified, or deleted files. You know, the delta. This means any files that are unchanged aren't uploaded when you run firebase deploy. You may not notice a big improvement in performance if your site is only a few files. However, it will make a huge difference for sites with a large amount of existing unmodified assets.

Make sure to get the newest version of the Firebase CLI (firebase-tools on npm) and you'll get super fast uploads!

[samtstern]

cc @Memeriaj

[mesqueeb]

@samtstern Thanks for your information on the .firebase folder. Do you know if we need to best commit this file to github in our projects? What would you do?

[samtstern]

It depends on how you deploy. If you deploy locally then no, you don't need to. If you deploy from a CI server then you'll want the server to have a cache of that folder somehow. Committing to source control is one way to do it, but not the best way.

…

On Tue, Oct 9, 2018, 7:29 PM Luca Ban ***@***.***> wrote: @samtstern <https://github.com/samtstern> Thanks for your information on the .firebase folder. Do you know if we need to best commit this file to github in our projects? What would you do? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIEw6u6AIxdXXUOApXwNohBgtddm48gLks5ujVuYgaJpZM4Igep8> .

[mbleigh]

FWIW the .firebase folder just speeds up the content hashing phase of the deploy, which usually isn't the majority of the time. You're probably fine without it, but caching it in your CI system will give you a little performance boost

…

On Tue, Oct 9, 2018, 7:40 PM Sam Stern ***@***.*** wrote: It depends on how you deploy. If you deploy locally then no, you don't need to. If you deploy from a CI server then you'll want the server to have a cache of that folder somehow. Committing to source control is one way to do it, but not the best way. On Tue, Oct 9, 2018, 7:29 PM Luca Ban ***@***.***> wrote: > @samtstern <https://github.com/samtstern> Thanks for your information on > the .firebase folder. Do you know if we need to best commit this file to > github in our projects? What would you do? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #133 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AIEw6u6AIxdXXUOApXwNohBgtddm48gLks5ujVuYgaJpZM4Igep8 > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAD_rdncPZfIR9IQDntJZQzx0JsSdMnks5ujV4agaJpZM4Igep8> .

[jcwrightson]

How does this handle *.rules files? Currently all my *.rules files continue to be uploaded and I've run into quota limits recently. As a work around I've edited firebase.json and removed these definitions in order to deploy without unnecessarily uploading these files.

[samtstern]

@jcwrightson this issue was only about Firebase Hosting and how it handles uploading web assets. De-duping *.rules files before uploading is a good idea! Do you mind filing a new Feature Request for that?

[jcwrightson]

@samtstern done! :)