Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: improve Reader docs (models) #5369

Open
anakin87 opened this issue Jul 14, 2023 · 3 comments
Open

Docs: improve Reader docs (models) #5369

anakin87 opened this issue Jul 14, 2023 · 3 comments
Labels
2.x Related to Haystack v2.0 P3 Low priority, leave it in the backlog type:documentation Improvements on the docs

Comments

@anakin87
Copy link
Member

(stems from #5340)

https://docs.haystack.deepset.ai/docs/reader

  • There is not a clear distinction between base models (to be fine-tuned) vs pretrained models (which can just be used out-of-the-box)
  • similar: no clear separation between using Reader to perform Question Answering and using it to fine-tune base models
  • The proposed pretrained models are outdated (we are not promoting our most recent QA models)

(Don't mean to be harsh, I'm just in a hurry. 😃 )

@anakin87 anakin87 added the type:documentation Improvements on the docs label Jul 14, 2023
@dfokina dfokina added the P1 High priority, add to the next sprint label Jul 20, 2023
@anakin87
Copy link
Member Author

Related Discord message: https://discord.com/channels/993534733298450452/1130211893508853861/1131319426180984863

To understand which the better Reader models are, we can ask Sol-Eng...

@anakin87
Copy link
Member Author

These benchmarks can also help: https://paperswithcode.com/sota/question-answering-on-squad-v2

@dfokina dfokina added P2 Medium priority, add to the next sprint if no P1 available and removed P1 High priority, add to the next sprint labels Aug 21, 2023
@dfokina dfokina closed this as not planned Won't fix, can't repro, duplicate, stale Dec 6, 2023
@anakin87
Copy link
Member Author

anakin87 commented Dec 7, 2023

Although it is not urgent, I think that providing some suggestions to users on models to use is still a good idea for 2.0.

This page of DC docs may serve as inspiration.

@anakin87 anakin87 reopened this Dec 7, 2023
@anakin87 anakin87 removed the P2 Medium priority, add to the next sprint if no P1 available label Dec 7, 2023
@masci masci added the P2 Medium priority, add to the next sprint if no P1 available label Dec 11, 2023
@masci masci added the 2.x Related to Haystack v2.0 label Jan 11, 2024
@masci masci added P3 Low priority, leave it in the backlog and removed P2 Medium priority, add to the next sprint if no P1 available labels Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 P3 Low priority, leave it in the backlog type:documentation Improvements on the docs
Projects
None yet
Development

No branches or pull requests

3 participants