-
Notifications
You must be signed in to change notification settings - Fork 698
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move model download from the Docker image build step to the run script #8
Comments
Completely agree with you here, @notocwilson! Thanks for outlining your thoughts. The main reason we went with the model image inside the Docker image was purely because of time constraints. We wanted to launch LlamaGPT as an Umbrel app, and because Umbrel users are mostly non-technical and install it from the app store, they won't be able to see the download progress of the model unless we make changes to the front-end showing the model download progress, and starting the API container once the it is downloaded. The goal is to:
|
Need help with this? Happy to take this one :). |
With the Docker files, as designed currently, you are staging a rather large file inside of a Docker image, which arguably violates the Docker Image best practice guideline of Don't install unnecessary packages. Ultimately, this dramatically inflates the size of the image to contain the model when it should be treated as a static asset that should be attached to a container.
By migrating the code to download the model to the run script, operators can now attach a volume to
/models
with storage that is defined outside of the context of the container. This could be a volume that lives on the same host, just as it did before if it was in the context of the Image, or could be provided by any other storage driver supported by Docker. Changes to the Docker Image will not require re-downloading and re-staging the model, significantly lowering both build time and bandwidth required to rebuild the image.In summary:
The text was updated successfully, but these errors were encountered: