Deprecate torchtext dependent models and related code. (#1831)

Summary:
torchtext is on a much lessen supported model, so we are deprecating the workload and make torchbench not depend on torchtext.

Fixes #1830

Pull Request resolved: #1831

Reviewed By: abhinavarora

Differential Revision: D48415274

Pulled By: xuzhao9

fbshipit-source-id: 9bb90ec3d84d2260b20cab1051d14c3d0a3129a0

.github/scripts/run.sh

@@ -1,6 +1,6 @@
 #!/bin/sh
 # This script runs TorchBench without installing the pytorch and torch dep packages
-# It assumes pytorch, torchtext, and torchvision have already been installed
+# It assumes pytorch and torchvision have already been installed
 # Usage:
 # run.sh RESULT_DIR [BENCHMARK_FILTER]
 # The RESULT_DIR is required, BENCHMARK_FILTER is optional

.github/workflows/gcp-a100-bisection.yml

@@ -40,12 +40,6 @@ jobs:
  repository: pytorch/vision
  path: srcs/vision
  fetch-depth: 0
- - name: Checkout torchtext
- uses: actions/checkout@v3
- with:
- repository: pytorch/text
- path: srcs/text
- fetch-depth: 0
  - name: Checkout torchdata
  uses: actions/checkout@v3
  with:

.github/workflows/pr-gpu-stability-ci.yml

@@ -33,7 +33,7 @@ jobs:
  conda install -y numpy requests=2.22 ninja pyyaml mkl mkl-include setuptools \
  cmake cffi typing_extensions future six dataclasses tabulate gitpython
  # Install pytorch nightly
- pip install --pre torch torchtext torchvision torchaudio \
+ pip install --pre torch torchvision torchaudio \
  -f https://download.pytorch.org/whl/nightly/${CUDA_VERSION}/torch_nightly.html
  # Install torchbench dependencies
  python install.py

.github/workflows/v3-bisection.yml

@@ -38,12 +38,6 @@ jobs:
  repository: pytorch/vision
  path: srcs/vision
  fetch-depth: 0
- - name: Checkout torchtext
- uses: actions/checkout@v3
- with:
- repository: pytorch/text
- path: srcs/text
- fetch-depth: 0
  - name: Checkout torchdata
  uses: actions/checkout@v3
  with:

README.md

@@ -26,14 +26,14 @@ If you are running NVIDIA GPU tests, we support CUDA 11.8+, and use CUDA 11.8 as
 conda install -y -c pytorch magma-cuda118
 ```
 
-Then install pytorch, torchtext, torchvision, and torchaudio using conda:
+Then install pytorch, torchvision, and torchaudio using conda:
 ```
-conda install pytorch torchvision torchtext torchaudio pytorch-cuda=11.8 -c pytorch-nightly -c nvidia
+conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch-nightly -c nvidia
 ```
 Or use pip:
 (but don't mix and match pip and conda for the torch family of libs! - [see notes below](#notes))
 ```
-pip install --pre torch torchvision torchtext torchaudio -i https://download.pytorch.org/whl/nightly/cu118
+pip install --pre torch torchvision torchaudio -i https://download.pytorch.org/whl/nightly/cu118
 ```
 
 Install the benchmark suite, which will recursively install dependencies for all the models. Currently, the repo is intended to be installed from the source tree.
@@ -44,9 +44,9 @@ python install.py
 ```
 
 ### Building From Source
-Note that when building PyTorch from source, torchtext, torchvision and torchaudio must also be built from source to make sure the C APIs match.
+Note that when building PyTorch from source, torchvision and torchaudio must also be built from source to make sure the C APIs match.
 
-See detailed instructions to install torchtext [here](https://github.com/pytorch/text), torchvision [here](https://github.com/pytorch/vision) and torchaudio [here](https://github.com/pytorch/audio).
+See detailed instructions to install torchvision [here](https://github.com/pytorch/vision) and torchaudio [here](https://github.com/pytorch/audio).
 Make sure to enable CUDA (by `FORCE_CUDA=1`) if using CUDA.
 Then,
 ```
@@ -58,8 +58,8 @@ python install.py
 ### Notes
 - Setup steps require network connectivity - make sure to enable a proxy if needed.
 - We suggest using the latest PyTorch nightly releases to run the benchmark. Stable versions are NOT tested or maintained.
-- torch, torchtext, torchvision, and torchaudio must all be installed from the same build process. This means it isn't possible to mix conda torchtext
- with pip torch, or mix built-from-source torch with pip torchtext. It's important to match even the conda channel (nightly vs regular).
+- torch, torchvision, and torchaudio must all be installed from the same build process. This means it isn't possible to mix conda torchvision
+ with pip torch, or mix built-from-source torch with pip torchvision. It's important to match even the conda channel (nightly vs regular).
  This is due to the differences in the compilation process used by different packaging systems producing incompatible Python binary extensions.
 
 ## Using a low-noise machine
@@ -84,7 +84,7 @@ There are multiple ways for running the model benchmarks.
 
 `userbenchmark` allows to develop and run customized benchmarks.
 
-In each model repo, the assumption is that the user would already have all of the torch family of packages installed (torch, torchtext, torchvision,...) but it installs the rest of the dependencies for the model.
+In each model repo, the assumption is that the user would already have all of the torch family of packages installed (torch, torchvision, torchaudio...) but it installs the rest of the dependencies for the model.
 
 ### Using `test.py`
 `python test.py` will execute the APIs for each model, as a sanity check. For benchmarking, use `test_bench.py`. It is based on unittest, and supports filtering via CLI.

bisection.py

@@ -1,7 +1,7 @@
 """bisection.py
 Runs bisection to determine PRs that trigger performance signals.
-It assumes that the pytorch, torchbench, torchtext, torchvision, and torchaudio repositories provided are all clean with the latest code.
-By default, the torchaudio, torchvision and torchtext packages will be fixed to the latest commit on the same pytorch commit date.
+It assumes that the pytorch, torchbench, torchvision, and torchaudio repositories provided are all clean with the latest code.
+By default, the torchaudio and torchvision packages will be fixed to the latest commit on the same pytorch commit date.
 
 Usage:
  python bisection.py --work-dir <WORK_DIR> \
@@ -53,11 +53,6 @@
  "url": "https://github.com/pytorch/vision.git",
  "build_command": [sys.executable, "setup.py", "install"],
  },
- "torchtext": {
- "name": "text",
- "url": "https://github.com/pytorch/text.git",
- "build_command": [sys.executable, "setup.py", "clean", "install"],
- },
  "torchaudio": {
  "name": "audio",
  "url": "https://github.com/pytorch/audio.git",

conftest.py

@@ -62,12 +62,6 @@ def pytest_benchmark_update_machine_info(config, machine_info):
  machine_info['pytorch_git_version'] = torch.version.git_version
  machine_info['cuda_version'] = torch.version.cuda
 
- try:
- import torchtext
- machine_info['torchtext_version'] = torchtext.__version__
- except ImportError:
- machine_info['torchtext_version'] = '*not-installed*'
-
  try:
  import torchvision
  machine_info['torchvision_version'] = torchvision.__version__

docker/gcp-a100-runner-dind.dockerfile

@@ -28,17 +28,6 @@ RUN cd /workspace && mkdir tmp_nvidia && cd tmp_nvidia && \
 
 # Source of the CUDA installation scripts:
 # https://github.com/pytorch/builder/blob/main/common/install_cuda.sh
-# Install CUDA 11.7 and cudnn 8.5.0.96
-RUN cd /workspace && wget -q https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run -O cuda_11.7.0_515.43.04_linux.run && \
- sudo bash ./cuda_11.7.0_515.43.04_linux.run --toolkit --silent && \
- rm -f cuda_11.7.0_515.43.04_linux.run
-RUN cd /workspace && wget -q https://ossci-linux.s3.amazonaws.com/cudnn-linux-x86_64-8.5.0.96_cuda11-archive.tar.xz -O cudnn-linux-x86_64-8.5.0.96_cuda11-archive.tar.xz && \
- tar xJf cudnn-linux-x86_64-8.5.0.96_cuda11-archive.tar.xz && \
- cd cudnn-linux-x86_64-8.5.0.96_cuda11-archive && \
- sudo cp include/* /usr/local/cuda-11.7/include && \
- sudo cp lib/* /usr/local/cuda-11.7/lib64 && \
- sudo ldconfig && \
- cd .. && rm -rf cudnn-linux-x86_64-8.5.0.96_cuda11-archive && rm -f cudnn-linux-x86_64-8.5.0.96_cuda11-archive.tar.xz
 
 # Install CUDA 11.8 and cudnn 8.7 and NCCL 2.15
 RUN cd /workspace && wget -q https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run -O cuda_11.8.0_520.61.05_linux.run && \

gen_summary_metadata.py

@@ -131,7 +131,7 @@ def _write_metadata_yaml_files(extracted_details: List[Tuple[str, Dict[str, Any]
  parser.add_argument("--eval-nograd", default=None, type=_parser_helper,
  help="Whether to enable no_grad during eval.")
  # parser.add_argument("--origin", default=None,
- # help="Location of benchmark's origin. Such as torchtext or torchvision.")
+ # help="Location of benchmark's origin. Such as torchaudio or torchvision.")
  # parser.add_argument("--train-dtype", default=None,
  # choices=['float32', 'float16', 'bfloat16', 'amp'], help="Which fp type to perform training.")
  # parser.add_argument("--eval-dtype", default=None,

requirements.txt

@@ -19,6 +19,5 @@ pyyaml
 numpy==1.21.2
 # Need https://github.com/kornia/kornia/commit/53808e5 to work on PyTorch nightly
 git+https://github.com/kornia/kornia.git@b7050c3
-scipy # for lazy_bench.py
 submitit
 pynvml