Add Generative QA Models like RAG

[tholor]

What?
So far most generative QA models were not really useful in practice, because they could only answer very generic questions that were included in their "wiki + web" training corpora. For use cases in the industry we mostly want to:

• ask domain-specific questions
• get the answer from a specified, reliable corpus
• have a possibility to check the answer (e.g. by inspecting the document / context where it came from)

Pure generative models don't fulfill these requirements. However, recent retrieval-augmented approaches could be interesting to test.

How?

The latest transformers release comes with the RAG model from Facebook (https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models). We could add a "TransformersGenerator" class in Haystack that gets documents from the retriever and generates the answer conditioned on those.

[Weilin37]

This would be really cool for domain-specific literature review

[nsankar]

@tholor As it is now in RAG, Is it correct to pass the haystack retriever output as inputs to RAG as cited in the snippet below in the RAG reference code given here ?

input_dict = tokenizer.prepare_seq2seq_batch(pass formatted haystack retriever outputs here, return_tensors="pt")
input_ids = input_dict["input_ids"]
outputs = model(input_ids=input_ids, labels=input_dict["labels"])

[tholor]

@nsankar I didn't have time yet to look into this, but I believe something like this should be possible with the RagTokenForGeneration class. Your "haystack retriever outputs" would be pretty much standard strings that we can pass there. Did you try it already?

[nsankar]

@tholor Yet to try this. I shall feedback once I try. Thank you for the inputs.

[Weilin37]

I'm here to cheerlead you guys on! Can't wait for this

[tholor]

Thanks to @lalitpagaria RAG integration in #484 is almost done!
We just need to do a FARM release before we can merge. The current plan is to do the release next week.

If you are very eager to try, you could already test it on the branch itself. There's already a small tutorial notebook!
Only be aware to run pip install -e . once to install the latest FARM version from master which is required for RAG.

[Weilin37]

@tholor I am definitely eager to try! Where is the tutorial notebook? I'll try it on the branch

[tholor]

@Weilin37 You can find the preliminary version here: https://github.com/lalitpagaria/haystack/blob/implement_RAG/tutorials/Tutorial7_RAG_Generator.ipynb

[lalitpagaria]

@Weilin37 In this notebook you need to use different haystack version.
update -

!pip install git+https://github.com/deepset-ai/haystack/
!pip install urllib3==1.25.4

to -

!pip install git+https://github.com/lalitpagaria/haystack.git@implement_RAG
!pip install urllib3==1.25.4

[Weilin37]

Thank you!

I am a "No module named 'haystack.generator'".
I guess this is the pip install -e . thing? However I seem to get an error when I run that command. Something about setup.py not found and directory cannot be installed in editable mode.

[lalitpagaria]

Please try this script https://github.com/lalitpagaria/haystack/blob/implement_RAG/tutorials/Tutorial7_RAG_Generator.py

Install pip install git+https://github.com/lalitpagaria/haystack.git@implement_RAG in separate virtualenv.

[tholor]

Fixed by #484