Added parallel calls to OpenAI with asyncio #21
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ganize duplicate code.
…is open, the more stuff I will add to it.
* SImplify utils to return lists instead of dictionary * A query has many retrieved documents and many answers. Not the opposite. * Tests for new query-document types * small todo for OpenAI LLM * Fixed reasoner evaluator and updated mocked llm_provider * Update custom prompt for retrieval evaluator * Update Domain Expert evaluator * rdnam evaluator as an example of meta_data usage. * Custom metadata in prompt * Cleanup rdnam * Start updating answer evaluator to same interface as retrieval. Re-organize duplicate code. * dummy build_message for domain expert * _build_message no longer abstract. * custom prompt for answer updated to new formatting * Better handling of parsed answers * Simplify skip_tuples * The great simplification of the Pairwise Answer Evaluator * test data column fixes * Readme update, new tests for the readme example. Other fixes. * More general JSON parsing, moving more stuff to BaseEvaluator * Removed unused file * Fix to output file * Fix to output file * Move to pydantic, batch_eval returns existing evaluations, fixes to output * Easier to pass retrieved documents to answer evaluator * removed unused tuple_columns * Solve pydantic warnings and formatting. All tests passing. * Solve pydantic warnings and formatting. All tests passing. * add pydantic as a requirement * Solve pydantic deprecation warnings * Fix import warning * version bump of pydantic to solve openai+mypy issues * More strict mypy checks * not-so-strict mypy. Mypy is happy. Pytest isn't. * Clean-up, fix tests, update readme. * Splitting configurations into multiple files. AKA: The longer the PR is open, the more stuff I will add to it.
* Supports pydantic v1 * Unify BaseModel interface, work around on args.py. * Change pydantic version to 1.9 on requirements and 2.7 on dev * Fix for type annotation on 1.9 * fix for 1.9 on args.py * fix for Pydantic 1.9 on args.py
* Create python-publish.yml * adapt workflow from https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#the-whole-ci-cd-workflow * forgot the package name --------- Co-authored-by: Fernando <[email protected]>
frejonb
approved these changes
Apr 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.