Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: load() missing 1 required positional argument: 'data_dir' #16

Closed
keherri opened this issue Jan 28, 2020 · 0 comments
Closed

Comments

@keherri
Copy link

keherri commented Jan 28, 2020

When trying to load a finetuned model I received the following error:

In [7]: reader = FARMReader(model_name_or_path="model_dir", use_gpu=True)                                                                                                      
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-44c12407fd4e> in <module>
----> 1 reader = FARMReader(model_name_or_path="model_dir", use_gpu=True)

~/SageMaker/haystack/haystack/reader/farm.py in __init__(self, model_name_or_path, context_window_size, no_ans_threshold, batch_size, use_gpu, n_candidates_per_passage)
     51 
     52 
---> 53         self.inferencer = Inferencer.load(model_name_or_path, batch_size=batch_size, gpu=use_gpu, task_type="question_answering")
     54         self.inferencer.model.prediction_heads[0].context_window_size = context_window_size
     55         self.inferencer.model.prediction_heads[0].no_ans_threshold = no_ans_threshold

~/SageMaker/FARM/farm/infer.py in load(cls, model_name_or_path, batch_size, gpu, task_type, return_class_probs, strict, max_seq_len)
    135                 processor = InferenceProcessor.load_from_dir(model_name_or_path)
    136             else:
--> 137                 processor = Processor.load_from_dir(model_name_or_path)
    138 
    139         # b) or from remote transformers model hub

~/SageMaker/FARM/farm/data_handler/processor.py in load_from_dir(cls, load_dir)
    188         del config["tokenizer"]
    189 
--> 190         processor = cls.load(tokenizer=tokenizer, processor_name=config["processor"], **config)
    191 
    192         for task_name, task in config["tasks"].items():

TypeError: load() missing 1 required positional argument: 'data_dir'

I was able to fix this by add "data_dir": "" to the processor_config.json file in the finetuned model directory. The file now looks like this :

{"baskets": [], "data_dir": "", "dev_filename": "dev-v2.0.json", "dev_split": 0, "doc_stride": 128, "max_query_length": 64, "max_seq_len": 256, "ph_output_type": "per_token_squad", "proxies": null, "target": "classification", "tasks": {"question_answering": {"label_list": ["start_token", "end_token"], "metric": "squad", "label_tensor_name": "question_answering_label_ids", "label_name": "question_answering_label", "label_column_name": null, "task_type": null}}, "test_filename": null, "train_filename": "train-v2.0.json", "tokenizer": "DistilBertTokenizer", "processor": "SquadProcessor"}
@keherri keherri closed this as completed Jan 28, 2020
masci pushed a commit that referenced this issue Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant