FileNotFoundError: [Errno 2] No such file or directory: 'roberta-base-squad2/processor_config.json'

[thaisnang]

For some reason, the config file is not getting dumped in the folder. I have tried changing folder permissions but no help.

[tanaysoni]

Hi @thaisnang, could you provide more details on what code/tutorial you're running and the full error stack trace that you get?

[thaisnang]

I ran tutorial 1 with FARMReader. It ran the first time then I tried again but this time with the offline model (same model base RoBERTa). After that it always gives the following:-
05/19/2020 14:16:31 - INFO - elasticsearch - PUT https://localhost:9200/document [status:400 request:0.004s]
05/19/2020 14:16:31 - INFO - haystack.indexing.io - Found data stored in data/article_txt_got. Delete this first if you really want to fetch new data.
05/19/2020 14:16:31 - INFO - elasticsearch - POST https://localhost:9200/_count [status:200 request:0.536s]

05/19/2020 14:16:52 - INFO - elasticsearch - POST https://localhost:9200/_bulk [status:200 request:1.665s]
05/19/2020 14:16:53 - INFO - elasticsearch - POST https://localhost:9200/_bulk [status:200 request:0.399s]
05/19/2020 14:16:53 - INFO - haystack.indexing.io - Wrote 517 docs to DB
05/19/2020 14:16:53 - INFO - farm.utils - device: cuda n_gpu: 1, distributed training: False, automatic mixed precision training: None
05/19/2020 14:17:04 - WARNING - farm.modeling.language_model - Could not automatically detect from language model name what language it is.
We guess it's an ENGLISH model ...
If not: Init the language model by supplying the 'language' param.
Traceback (most recent call last):
File "Tutorial1_Basic_QA_Pipeline.py", line 123, in
reader = FARMReader(model_name_or_path="roberta-base-squad2", use_gpu=True)
File "/home/imsai/.local/lib/python3.6/site-packages/haystack/reader/farm.py", line 86, in init
doc_stride=doc_stride, num_processes=num_processes)
File "/home/imsai/.local/lib/python3.6/site-packages/farm/infer.py", line 194, in load
processor = Processor.load_from_dir(model_name_or_path)
File "/home/imsai/.local/lib/python3.6/site-packages/farm/data_handler/processor.py", line 182, in load_from_dir
config = json.load(open(processor_config_file))
FileNotFoundError: [Errno 2] No such file or directory: 'roberta-base-squad2/processor_config.json'

---

I tried with transformer as well it gives the following--

05/19/2020 14:22:30 - INFO - elasticsearch - PUT https://localhost:9200/document [status:400 request:0.036s]
05/19/2020 14:22:30 - INFO - haystack.indexing.io - Found data stored in data/article_txt_got. Delete this first if you really want to fetch new data.
05/19/2020 14:22:30 - INFO - elasticsearch - POST https://localhost:9200/_count [status:200 request:0.004s]
05/19/2020 14:22:30 - INFO - haystack.indexing.io - Skip writing documents since DB already contains 517 docs ... (Disable only_empty_db, if you want to add docs anyway.)
05/19/2020 14:22:38 - INFO - elasticsearch - POST https://localhost:9200/document/_search [status:200 request:0.318s]
05/19/2020 14:22:38 - INFO - haystack.retriever.elasticsearch - Got 10 candidates from retriever
05/19/2020 14:22:38 - INFO - haystack.finder - Reader is looking for detailed answer in 362347 chars ...
convert squad examples to features: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 7.14it/s]
add example index and unique id: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 5629.94it/s]
Traceback (most recent call last):
File "Tutorial1_Basic_QA_Pipeline.py", line 140, in
prediction = finder.get_answers(question="Who is the father of Arya Stark?", top_k_retriever=10, top_k_reader=5)
File "/home/imsai/.local/lib/python3.6/site-packages/haystack/finder.py", line 45, in get_answers
top_k=top_k_reader)
File "/home/imsai/.local/lib/python3.6/site-packages/haystack/reader/transformers.py", line 77, in predict
predictions = self.model(query, topk=self.n_best_per_passage)
File "/home/imsai/.local/lib/python3.6/site-packages/transformers/pipelines.py", line 1042, in call
for s, e, score in zip(starts, ends, scores)
File "/home/imsai/.local/lib/python3.6/site-packages/transformers/pipelines.py", line 1042, in
for s, e, score in zip(starts, ends, scores)
KeyError: 0

[thaisnang]

Note:- I did not download the RoBERTa separately I just renamed the files from the cache it automatically downloaded. I have renamed them properly that I am sure of. Hopefully, this is not affecting it.

[tanaysoni]

Hi @thaisnang, by default, the models are cached and are not re-downloaded on every execution. If that doesn't fit your workflow, I am curious to know more on how you plan to use the save(offline) functionality.

Here's how you can save the model of a FARMReader:

reader.inferencer.save("path-to-save")

and load it again by supplying the path:

reader = FARMReader(model_name_or_path="path-to-save")

[thaisnang]

Actually I saw the model was downloading again when I ran it the second time. So I thought instead of downloading every execution why don't I just copy the cached model and properly rename it and use it as an offline model. And that's what I did, it should not interfere with the function right?

[thaisnang]

OK, I downloaded again and this time the model did not redownload it was using the cached model.
And the model was saved as well. Thanks.