RealSI: Open Benchmark for Simultaneous Interpretation in Real-world Scenarios

For details of the dataset, please refer to the Technical Report of Cross Language Agent-Simultaneous Interpretation, CLASI.

Please follow the instructions to download the dataset.

pip3 install -r requirements.txt
mkdir data/en2zh/audio data/zh2en/audio

python3 toolkits/download_audio.py

RealSI is a benchmark for Simultaneous Interpretation(SI) in real-world scenarios. This project intends to release a public test set for SI, in order to evaluate the speech translation performance of the model in difficult scenarios such as long audio, speakers not being fully prepared, and heavy accents.

Audio data are collected from public videos, covering 2 languages and 10 different topics. We cut out video clips of 3-7 minutes, and label the timestamps, transcripts and translations of speech in the video.

DISCLAIMER: We do not own the copyright of the videos and only release our annotation together with the publicly available website links of the corresponding videos. If anyone believes that the content constitutes infringement, please contact us. We will remove the relevant content as soon as possible once confirmed. Any content in this dataset is available for educational and informational purposes only. You are solely responsible for legal liability arising from your improper use of the dataset content. Refer to License

We save each audio in separate files, each file contains information of id and duration of the audio. The utterances are grouped in semantic segments, which convey complete semantic for translation. When conducting human evaluation, we recommend evaluating the model prediction results according to this segmentation.

We show the data structure by taking en2zh-01-tech as example.

{
  "vid": "en2zh-01-tech",
  "duration": 205000,
  "segment": [
    {
      "start_time": 0,
      "end_time": 5790,
      "utterance": [
        {
          "sub_start_time": 0,
          "sub_end_time": 2810,
          "sub_src_text": "Round-robin um um load balancing scheme,",
          "sub_trg_text": "轮询负载均衡方案。",
          "term": [
            {
                "src": "Round-robin",
                "trg": "轮询"
            },
            ... ...
          ]
        },
        ... ...
      ],
    },
    ... ...
  ]
}

This dataset is released under the Creative Commons Attribution 4.0 International License. Please refer to CC-BY-4.0 for more details.