unable to use whisper offline

[vrgovinda]

Hi,
I am using the latest version of whisper updated today using pip install -U openai-whisper.
I have the downloaded the small, small.en, tiny, tiny.en, base, base.en and stored it in a folder which I mention using --model-dir as you can see below.

But ever since I installed it [around 3 months ago], I always had to connect to the internet to do the transcription for the first time. Subsequent usage work offline.

How can I use whisper purely offline?

I invoke whisper using

whisper --model base.en --model_dir /home/vrgovinda/offline-backup/transcription-seva/whisper-models/ --output_dir /home/vrgovinda/Documents/sevā/Transcription_BRSM_lectures/openAI-whisper/transcripts/ --language English 3*.mp3

I get a big error list:

/home/vrgovinda/.local/lib/python3.9/site-packages/whisper/transcribe.py:114: UserWarning: FP16 is not supported on CPU; using FP32 instead
  warnings.warn("FP16 is not supported on CPU; using FP32 instead")
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f4267b745b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))

The error in question is this:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /gpt-2/encodings/main/vocab.bpe (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4267b745b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

The above error indicates that the program is trying to connect to the website openaipublic.blob.core.windows.net

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vrgovinda/.local/bin/whisper", line 8, in <module>
    sys.exit(cli())
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/whisper/transcribe.py", line 437, in cli
    result = transcribe(model, audio_path, temperature=temperature, **args)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/whisper/transcribe.py", line 142, in transcribe
    tokenizer = get_tokenizer(model.is_multilingual, language=language, task=task)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/whisper/tokenizer.py", line 385, in get_tokenizer
    encoding = get_encoding(name=encoding_name)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/whisper/tokenizer.py", line 355, in get_encoding
    pat_str=gpt2()["pat_str"],
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/tiktoken_ext/openai_public.py", line 11, in gpt2
    mergeable_ranks = data_gym_to_mergeable_bpe_ranks(
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/tiktoken/load.py", line 73, in data_gym_to_mergeable_bpe_ranks
    vocab_bpe_contents = read_file_cached(vocab_bpe_file).decode()
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/tiktoken/load.py", line 46, in read_file_cached
    contents = read_file(blobpath)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/tiktoken/load.py", line 24, in read_file
    return requests.get(blobpath).content
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/vrgovinda/.local/lib/python3.9/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: 

This is again reflected in this error:

HTTPSConnectionPool(host='openaipublic.blob.core.windows.net',port=443): Max retries exceeded with url: /gpt-2/encodings/main/vocab.bpe (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4267b745b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

[0xbytejay]

@vrgovinda
Specifying the language as 'en' and modifying the 'vocab_bpe_file' and 'encoder_json_file' in the 'gpt2' function of 'openai_public.py' to local paths can avoid this issue.

transcribe = model.transcribe(audio=path,language="en")

python3.9/site-packages/tiktoken_ext/openai_public.py:

def gpt2():
    mergeable_ranks = data_gym_to_mergeable_bpe_ranks(
        vocab_bpe_file="/root/workspace/whisper/vocab.bpe",
        encoder_json_file="/root/workspace/whisper/encoder.json",
    )

[vrgovinda]

@0xbytejay

Thanks a lot Jay! That really gave me some insight.
Now, my next questions are:

1. from where can I get "vocab.bpe" and "encoder.json"? A local search in my machine did yield any result.
2. I have mentioned the language as "English"

whisper --model base.en --model_dir /path/to/model --output_dir /path/to/output --language English filename.mp3

Is this still a problem? Should I specify the language as "en" only.
P. S.: I am using the CLI not a python IDE.

Thanks again.

[nicholasgcotton]

The “currnent” online locations for those files are listed in the code we need to change at lines 12-13 in python3.9/site-packages/tiktoken_ext/openai_public.py.

I am note quite sure how to translate “ /root/workspace/whisper/vocab.bpe” into a Windows file path though.

[nicholasgcotton]

I downloaded the files from the URLs in openai_public.py and moved them to C:\Users[Username].cache\whisper\ and rewrote the above code with C:/Users/[Username]/.cache/whisper/vocab.bpe and /encoder.json and then got an error that local ingest required the pip blobfile util, after going online temporarily for “pip install blobfile” the code subsequently worked offline again.

[nicholasgcotton]

As of some recent change Whisper now calls to the internet for vocab files. Downloading the vocab.bpe and encoder.json files from the URLs specified in openai_public.py and updating some local code will re-enable offline usage.
- Download Vocab.bpe
- Download Encoder.json
- Install the files to a folder of your choosing, for ease of use I recommend C:\Users[username]\.cache\whisper e.g. C:\Users\nic\.cache\whisper.
- You will then need to change your local copy of openai_public.py which will be installed in your python folder e.g. python3.9/site-packages/tiktoken_ext/openai_public.py to point to where you downloaded the files. Change lines 11 and 12 to remove the URL "https://openaipublic.blob.core.windows.net/gpt-2/encodings/main/" and replace it with your local copy, e.g. "C:/Users/nic/.cache/whisper"
- An extra utility is required to open the local files. Install the pip utility "blobfile". This can be done while online by running "pip install blobfile", but once that is done the code will subsequently run offline.

[Sethecientos]

You saved my life. Thanks

[vrgovinda]

Thanks @nicholasgcotton. Marked your comment as ANSWER.

I have another query though.
In openai_public.py please look through lines 26,39,52,65.
They also seem to contain some website links. How come this doesn't affect the offline usage? Just curious!