Self correction loop for RAG when "answer is not possible given context"

[Timoeller]

Is your feature request related to a problem? Please describe.
In RAG it can happen that requested information is not present in the given context documents.
Then the generative model is instructed to say: "answer is not possible given context".
But maybe the answer is present in documents that were not given to the generative model.

Please also look at deepset-ai/haystack-demos#23 and their insights into what is missing for implementing looping in a 2.0 pipeline.

Describe the solution you'd like
A loop passing retrieved documents, in order of relevance, until an answer is found or another stopping criterion is met.

Additional context
This feature is important to showcase how loops enhance RAG in a meaningful way.

[DomEscobar]

The challenge is rly if we have to iterate over all documents or if the confidence is high enough to answer and stop🤔

[Timoeller]

Good point.

I agree that iterating over all documents is not something you want to do in any realistic scenario. We will have a max_iterations counter that stops the looping.

Relying on the confidence score to stop might be tricky since neither sparse, dense, or a combination thereof produces meaningful confidence scores to the best of my knowledge. You can easily get stuck in endless loops then. Or do you know more on this?

[TuanaCelik]

Hey @Timoeller and @vblagoje to me it sounds like these 2 issues are solving similar situations: #6109

I may be totally off. But I want to add my 2 cents here too just in case.
I think the global issue we're trying to solve is having a 'decision' node, or a 'router' as Vladimir puts it. What happens based on that decision should be up to the user. For example, I'm trying to build a demo where 'if the answer is not provided, do websearch'. For Timo's scenario it's 'try again' and so on.

Would it make sense to combine these 2 efforts? Seems to me they might be going in a similar direction, with different implementation.

[ZanSara]

I think this one kind of depends on @vblagoje's issue. This one is oriented towards making a running example showcasing this feature, so imho it makes sense to keep both. Of course I'm gonna use the feature from that PR rather than implementing my own competing solution.

[TuanaCelik]

If it's any help, here's my implementation: https://colab.research.google.com/drive/1LSrAnsL5yLzmrXE1JVBdagPLCUHEOPKO?usp=sharing