Split pipeline tests into three suites #1755
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
julian-risch
approved these changes
Nov 15, 2021
There was a problem hiding this comment.
Just rename one of the tests that is currently called test_document_search_pipeline -> test_most_similar_documents_pipeline because it contains a MostSimilarDocumentsPipeline instead of a DocumentSearchPipeline. The problem existed before your changes already. Otherwise looks good to me. All tests accounted for. 👍
|
Somehow renaming that test uncovered an issue with it. It was probably never collected by pytest due to its name and never run 😱 Now fixing it, it should be just a naming issue. If it works out I'll then merge. |
tholor
added a commit
that referenced
this pull request
Nov 23, 2021
* adding yaml functionality to BaseStandardPipeline fixes #1681 * Add latest docstring and tutorial changes * Update API Reference Pages for v1.0 (#1729) * Create new API pages and update existing ones * Create query classifier page * Remove Objects suffix * Change answer aggregation key to doc_id, query instead of label_id, query (#1726) * Add debugging example to tutorial (#1731) * Add debugging example to tutorial * Add latest docstring and tutorial changes * Remove Objects suffix * Add latest docstring and tutorial changes * Revert "Remove Objects suffix" This reverts commit 6681cb0. * Revert unintentional commit * Add third debugging option * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix another self.device/s typo (#1734) * Fix yet another self.device(s) typo * Add typing to 'initialize_device_settings' to try prevent future issues * Fix bug in Tutorial5 * Fix the same bug in the notebook Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * added test for saving and loading prebuilt pipelines * fixed typo, changed variable name and added comments * Add latest docstring and tutorial changes * Fix a few details of some tutorials (#1733) * Make Tutorial10 use print instead of logs and fix a typo in Tutoria15 * Add a type check in 'print_answers' * Add same checks to print_documents and print_questions * Make RAGenerator return Answers instead of dictionaries * Fix RAGenerator tests Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix `print_answers` (#1743) * Fix a specific path of print_answers that was assuming answers are dictionaries Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Split pipeline tests into three suites (#1755) * Split pipeline tests into three suites * Will this trigger the CI? * Rename duplicate test into test_most_similar_documents_pipeline * Fixing a bug that was probably never noticed * Capitalize starting letter in params (#1750) * Capitalize starting letter in params Capitalized the starting letter in code examples for params in keeping with the latest names for nodes where first letter is capitalized. Refer: #1748 * Update standard_pipelines.py Capitalized some starting letters in the docstrings in keeping with the updated node names for standard pipelines * Multi query eval (#1746) * add eval() to pipeline * Add latest docstring and tutorial changes * support multiple queries in eval() * Add latest docstring and tutorial changes * keep single query test * fix EvaluationResult node_results default * adjust docstrings * Add latest docstring and tutorial changes * minor improvements from comments * Add latest docstring and tutorial changes * move EvaluationResult and calculate_metrics to schema * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Split summarizer tests in order to make windows CI work again (#1757) * separate testfile for summarizer with translation * Add latest docstring and tutorial changes * import SPLIT_DOCS from test_summarizer * add workflow_dispatch to windows_ci * add worflow_dispatch to linux_ci Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix import of EvaluationResult in test case * exclude test_summarizer_translation.py for windows_ci (#1759) * Pipelines now tolerate custom _debug content (#1756) * Pipelines now tolerate custom _debug content * Support Tables in all DocumentStores (#1744) * Add support for tables in SQLDocumentStore, FAISSDocumentStore and MilvuDocumentStore * Add support for WeaviateDocumentStore * Make sure that embedded meta fields are strings + add embedding_dim to WeaviateDocStore in test config * Add latest docstring and tutorial changes * Represent tables in WeaviateDocumentStore as nested lists * Fix mypy Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Allow TableReader models without aggregation classifier (#1772) * Fix usage of filters in `/query` endpoint in REST API (#1774) * WIP filter refactoring * fix filter formatting * remove inplace modification of filters * Public demo (#1747) * Queries now run only when pressing RUN. File upload hidden. Question is not sent if the textbox is empty. * Add latest docstring and tutorial changes * Tidy up: remove needless state, add comments, fix minor bugs * Had to add results to the status to avoid some bugs in eval mode * Added 'credits' * Add footers, update requirements, some random questions for the evaluation * Add requested changes * Temporary rollback the UI to the old GoT dataset Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Facilitate concurrent query / indexing in Elasticsearch with dense retrievers (new `skip_missing_embeddings` param) (#1762) * Filtering records not having embeddings * Added support for skip_missing_embeddings Flag. Default behavior is throw error when embeddings are missing. If skip_missing_embeddings=True then documents without embeddings are ignored for vector similarity * Fix for below error: haystack/document_stores/elasticsearch.py:852: error: Need type annotation for "script_score_query" * docstring for skip_missing_embeddings parameter * Raise exception where no documents with embeddings is found for Embedding retriever. * Default skip_missing_embeddings to True * Explicitly check if embeddings are present if no results are returned by EmbeddingRetriever for Elasticsearch * Added test case for based on Julian's input * Added test case for based on Julian's input. Fix pytest error on the testcase * Added test case for based on Julian's input. Fix pytest error on the testcase * Added test case for based on Julian's input. Fix pytest error on the testcase * Simplify code by using get_embed_count * Adjust docstring & error msg slightly * Revert error msg Co-authored-by: Malte Pietsch <[email protected]> * Huggingface private model support via API tokens (FARMReader) (#1775) * passed kwargs to model loading * Pass Auth token explicitly * add use_auth_token to get_language_model_class * added use_auth_token parameter at FARMReader * Add latest docstring and tutorial changes * added docs for parameter `use_auth_token` * Add latest docstring and tutorial changes * adding docs link * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * private hugging face models for retrievers (#1785) * private dpr * Add latest docstring and tutorial changes * added parameters to child functions * Add latest docstring and tutorial changes * added tableextractor * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * ignore empty filters parameter (#1783) * ignore empty filters parameter * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * initialize doc store with doc and label index in tutorial 5 (#1730) * initialize doc store with doc and label index * change ipynb according to py for tutorial 5 * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Small fixes to the public demo (#1781) * Make strealit tolerant to haystack not knowing its version, and adding special error for docstore issues * Add workaround for a Streamlit bug * Make default filters value an empty dict * Return more context for each answer in the rest api * Make the hs_version call not-blocking by adding a very quick timeout * Add disclaimer on low confidence answer * Use the no-answer feature of the reader to highlight questions with no good answer * Upgrade torch to v1.10.0 (#1789) * Upgrade torch to v1.10.0 * Adapt torch version for torch-scatter in TableQA tutorial * Add latest docstring and tutorial changes * Make torch version more flexible Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * adding yaml functionality to BaseStandardPipeline fixes #1681 * Add latest docstring and tutorial changes * added test for saving and loading prebuilt pipelines * fixed typo, changed variable name and added comments * Add latest docstring and tutorial changes * fix code rendering for example * Add latest docstring and tutorial changes Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Branden Chan <[email protected]> Co-authored-by: Julian Risch <[email protected]> Co-authored-by: Sara Zan <[email protected]> Co-authored-by: nishanthcgit <[email protected]> Co-authored-by: tstadel <[email protected]> Co-authored-by: bogdankostic <[email protected]> Co-authored-by: Malte Pietsch <[email protected]> Co-authored-by: C V Goudar <[email protected]> Co-authored-by: Kristof Herrmann <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trying to make pipeline tests run again on the CI