Releases: google/caliban
0.4.1: incremental JOSS release
0.4.0: Native MLFlow tracking server support
The biggest feature in this new release is native support for logging to an MLFlow tracking server using the UV Metrics project, via #35) This feature is in alpha, and baking internally; expect documentation soon!
If you want to try this out, check out the tutorial in tutorials/uv-metrics
: https://github.com/google/caliban/blob/master/tutorials/uv-metrics/README.md
More features
- minor bugfixes for GKE (#85)
- additional tests for gke.{types, util} (#84)
- re-order custom apt packages before pip requirements (#82)
- modify base image to our more general cloudbuild naming scheme (#80)
- updated
google-auth
dependency version to1.19.0
(#79) - add clearer contribution info (#76)
- Update uv-metrics tutorial (#74, #72)
- add support for running an embedded cloudsql_proxy (#60)
- bugfix for #65: do not add resource maxima when quota is < 1 (#67)
- Updated accelerator regions (and globally availabe AI Platform regions to
match the current state here):
https://cloud.google.com/ai-platform/training/docs/regions
0.3.0: Simpler cloud auth, custom base images, DLVMs
This release was focused on making it easier for others to contribute. The highlights are:
- we now support custom base images
- you no longer need a service account key to submit jobs to AI Platform
- we won't push anymore if a docker image already exists in Cloud, saving you some time and stdout spray
- custom base images via
.calibanconfig.json
, AND support, with special base image names, for all of Google's "Deep Learning VMs" instead of Caliban's default base images.
Thanks to @ramasesh , @eschnett, @ajslone and @sagravat for their contributions on this release!
For more detail, here's the CHANGELOG:
- @ramasesh Added a fix that prevented
pip
git dependencies from working in
caliban shell
mode (#55) This adds a
small update to the base image, so be sure to run
docker pull gcr.io/blueshift-playground/blueshift:cpu
docker pull gcr.io/blueshift-playground/blueshift:gpu
to get access to this fix.
-
Thanks to @eschnett,
--docker_run-args
can now deal with arbitrary
whitespace in the list of arguments, instead of single spaces only.
(#46) -
Caliban now authenticates AI Platform job submissions using the authentication
provided bygcloud auth login
, rather than requiring a service account key.
This significantly simplifies the setup required for a first time user. -
caliban cloud
now checks if the image exists remotely before issuing a
docker push
command on the newly built image
(#36) -
Big internal refactor to make it easier to work on code, increase test
coverage, add new backends (#32) -
add
schema
validation for.calibanconfig.json
. This makes it much easier
to add configuration knobs: #37 -
Custom base image support (#39), thanks
to #20 from @sagravat.
.calibanconfig.json
now supports a"base_image"
key. For the value, can
supply:- a Docker base image of your own
- a dict of the form
{"cpu": "base_image", "gpu": "base_image"}
with both
entries optional, of course.
Two more cool features.
First, if you use a format string, like
"my_image-{}:latest"
, the format
block{}
will be filled in with eithercpu
orgpu
, depending on the mode
Caliban is using.Second, we now have native support for Google's Deep Learning
VMs
as base images. The actual VM containers live
here.
If you provide any of the following strings, Caliban will expand them out to
the actual base image location:
dlvm:pytorch-cpu
dlvm:pytorch-cpu-1.0
dlvm:pytorch-cpu-1.1
dlvm:pytorch-cpu-1.2
dlvm:pytorch-cpu-1.3
dlvm:pytorch-cpu-1.4
dlvm:pytorch-gpu
dlvm:pytorch-gpu-1.0
dlvm:pytorch-gpu-1.1
dlvm:pytorch-gpu-1.2
dlvm:pytorch-gpu-1.3
dlvm:pytorch-gpu-1.4
dlvm:tf-cpu
dlvm:tf-cpu-1.0
dlvm:tf-cpu-1.13
dlvm:tf-cpu-1.14
dlvm:tf-cpu-1.15
dlvm:tf-gpu
dlvm:tf-gpu-1.0
dlvm:tf-gpu-1.13
dlvm:tf-gpu-1.14
dlvm:tf-gpu-1.15
dlvm:tf2-cpu
dlvm:tf2-cpu-2.0
dlvm:tf2-cpu-2.1
dlvm:tf2-cpu-2.2
dlvm:tf2-gpu
dlvm:tf2-gpu-2.0
dlvm:tf2-gpu-2.1
dlvm:tf2-gpu-2.2
Format strings work here as well! So, "dlvm:pytorch-{}-1.4"
is a totally valid
base image.
0.2.6: interactive progress bar support, better `stdout` handling
This release:
- Prepared for a variety of base images by setting up a cloud build matrix:
#25 - Added better documentation for
gcloud auth configure-docker
#26 - Added
close()
toTqdmFile
, preventing an error when pipingstdout
:
#30 tqdm
progress bars and other interactive outputs now display correctly in
caliban run
outputs.stdout
flushes properly! Before these changes,
stderr
would appear before anystdout
, making it difficult to store the
logs in a text file. Now, by default, python processes launched bycaliban run
won't buffer. #31
Thanks to @dthiagarajan for reporting an issue that led to this work.
interactive apt-get package installation support
This release:
- fixes the python binary that caliban notebook points to (now that we use conda)
- adds
DEBIAN_FRONTEND=noninteractive
to the apt-get command, so that packages liketexlive
won't freeze and wait for you to specify a timezone.
This makes it easy to add, for example, npm
and latex support to your caliban notebook
invocations.
Enjoy!
Bugfix release
google-auth
pushed an upgrade that broke an internal method we were using. This release pins the version and fixes the error.