Google protobuf package must be <= 3.20.1

[ugermann]

🐛 Bug

The current latest version of Google's protobuf python package is 4.x.x. However, the unbabel-comet python package requires (but does not specify) protobuf<=3.20.1.

To Reproduce

Here's a Dockerfile (for GPU-enabled nvidia-docker):
unbabel-comet-dockerfile.txt

ARG DISTRO=ubuntu20.04
FROM nvidia/cuda:11.4.0-runtime-$DISTRO
ENV DEBIAN_FRONTEND noninteractive
ENV distro=ubuntu2004

# see https://forumas.developer.nvidia.com/t/18-04-cuda-docker-image-is-broken/212892/9
RUN rm -f /etc/apt/sources.list.d/cuda.list
RUN rm -f /etc/apt/sources.list.d/nvidia-ml.list
RUN apt-key del 7fa2af80
RUN apt-get update && apt-get install -y --no-install-recommends wget
RUN wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/x86_64/cuda-keyring_1.0-1_all.deb
RUN dpkg -i cuda-keyring_1.0-1_all.deb

RUN apt update
RUN apt upgrade -y
RUN apt install -y python3
RUN apt install -y python3-pip
RUN pip install unbabel-comet
# RUN pip install protobuf==3.20.1

Build a Docker image:

docker build -t unbabelcomet:latest -f unbabel-comet-dockerfile.txt .

If you then run docker run --rm unbabelcomet:latest comet-score, you get the following:

Traceback (most recent call last):                                                                                                                                           
  File "/usr/local/bin/comet-score", line 5, in <module>                                                                                                                     
    from comet.cli.score import score_command                                                                                                                                
  File "/usr/local/lib/python3.8/dist-packages/comet/__init__.py", line 19, in <module>                                                                                      
    from .download_utils import download_model                                                                                                                               
  File "/usr/local/lib/python3.8/dist-packages/comet/download_utils.py", line 26, in <module>                                                                                
    from comet.models import available_metrics                                                                                                                               
  File "/usr/local/lib/python3.8/dist-packages/comet/models/__init__.py", line 17, in <module>                                                                               
    from .regression.regression_metric import RegressionMetric                                                                                                               
  File "/usr/local/lib/python3.8/dist-packages/comet/models/regression/regression_metric.py", line 26, in <module>                                                           
    from comet.models.base import CometModel                                                                                                                                 
  File "/usr/local/lib/python3.8/dist-packages/comet/models/base.py", line 28, in <module>                                                                                   
    import pytorch_lightning as ptl
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/__init__.py", line 30, in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/callbacks/__init__.py", line 26, in <module>
    from pytorch_lightning.callbacks.pruning import ModelPruning
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/callbacks/pruning.py", line 31, in <module>
    from pytorch_lightning.core.lightning import LightningModule
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/core/__init__.py", line 16, in <module>
    from pytorch_lightning.core.lightning import LightningModule
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/core/lightning.py", line 40, in <module>
    from pytorch_lightning.loggers import LightningLoggerBase
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/loggers/__init__.py", line 18, in <module>
    from pytorch_lightning.loggers.tensorboard import TensorBoardLogger
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/loggers/tensorboard.py", line 26, in <module>
    from torch.utils.tensorboard import SummaryWriter
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/tensorboard/__init__.py", line 13, in <module>
    from .writer import FileWriter, SummaryWriter  # noqa: F401
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/tensorboard/writer.py", line 9, in <module>
    from tensorboard.compat.proto.event_pb2 import SessionLog
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/compat/proto/event_pb2.py", line 17, in <module>
    from tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/compat/proto/summary_pb2.py", line 17, in <module>

Uncomment the last line in the Dockerfile, rebuild the Docker image and things work as expected:

Global seed set to 12
usage: comet-score [-h] [-s SOURCES] [-t TRANSLATIONS [TRANSLATIONS ...]]
                   [-r REFERENCES] [-d SACREBLEU_DATASET]
                   [--batch_size BATCH_SIZE] [--gpus GPUS] [--quiet]
                   [--accelerator {dp,ddp}] [--to_json TO_JSON]
                   [--model MODEL] [--model_storage_path MODEL_STORAGE_PATH]
                   [--mc_dropout MC_DROPOUT]
                   [--seed_everything SEED_EVERYTHING]
                   [--num_workers NUM_WORKERS] [--disable_bar]
                   [--disable_cache] [--disable_length_batching]
                   [--print_cache_info]
comet-score: error: You must specify a source (-s) or a sacrebleu dataset (-d)

Environment

OS: Ubuntu 20.04 with Nvidia docker (see https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker)
Packaging: Docker

[ricardorei]

Thanks for reporting this. This is a very recent bug on PyTorch lightning side: Lightning-AI/pytorch-lightning#13177

I'll pin that version on COMET side also while we wait for a new version in lightning

[Maaalik]

Thank you very much, @ugermann!
I am a noob, so I struggled a little with the minor typos in the commands, those ended up working for me:
$ docker build . -t unbabelcomet:latest -f unbabel-comet-dockerfile.txt
(i.e. with a dot after "build");
$ docker run --rm unbabelcomet:latest comet-score
(i.e. with two hyphens before the options "rm").
@ugermann, would there be a way to make this work in a Google Colab notebook ?

[ricardorei]

I released a new version that Pins this version. Nonetheless, it still looks like it is failing for Python 3.9.

In any case please test v1.1.1.

[ugermann]

@Maaalik Thanks for pointing out the errors in the docker commands I provided. They have now been corrected. I haven't used Google Colab yet, so I'm afraid I won't be able to provide any useful guidance in this respect.

[Maaalik]

Thank you very much, @ricardorei!
"comet==1.1.1" seems to be working on Google Colab notebooks after the change you made.
I will hold on to that docker image, though, you never know :) ! (Thank you again, @ugermann!)
Here is a link to a bare-bones Colab notebook that follows parts of the COMET readme tutorial :
https://colab.research.google.com/drive/1laUitV3nwcMme2vgiqd8IXdPLLY7VpAi?usp=sharing