Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge from main #10

Merged
merged 112 commits into from
Jun 26, 2024
Merged
Changes from 1 commit
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
9a8026c
feat(tests): add requirements-dev.txt
May 21, 2024
35dcaed
feat(tests): BaseIntegrationTestWithCache
May 21, 2024
a1c8491
feat(tests): ignore assert in ruff
May 21, 2024
6f1c120
feat(tests): TestIntroIntegration
May 21, 2024
b773796
feat(tests): TestIntroIntegration
May 21, 2024
32b5470
feat(tests): TestIntroIntegration
May 21, 2024
2e10d67
feat(tests): add integration test to CI.
May 21, 2024
6df93fc
feat(tests): add integration test to CI.
May 21, 2024
195cd03
feat(tests): add integration test to CI.
May 21, 2024
965fbf3
feat(tests): add integration test to CI.
May 21, 2024
ff075fe
feat(tests): add integration test to CI.
May 21, 2024
56b00ad
feat(tests): add integration test to CI.
May 21, 2024
742eb56
feat(tests): add pytest.ini for integration
May 25, 2024
598f831
feat(tests): add pytest.ini for integration
May 25, 2024
73ff317
feat(tests): change it to ci way -_-
May 25, 2024
f6a64e3
feat(tests): change it to ci way -_-
May 25, 2024
e593154
feat(tests): change it to ci way -_-
May 25, 2024
35c39ad
feat(tests): change it to ci way -_-
May 25, 2024
065f2d1
feat(tests): change it to ci way -_-
May 25, 2024
81851d1
feat(tests): change it to ci way -_-
May 25, 2024
f7fa698
feat(tests): change it to ci way -_-
May 25, 2024
3b7da8d
feat(tests): change it to ci way -_-
May 25, 2024
11c84bb
feat(tests): change it to ci way -_-
May 25, 2024
d96c0ce
feat(tests): change it to ci way -_-
May 25, 2024
b4f3391
feat(tests): change it to ci way -_-
May 25, 2024
0098e4e
feat(tests): cache file.
May 25, 2024
8f81e6c
feat(tests): cache file.
May 25, 2024
b2e51bb
feat(tests): cache file.
May 25, 2024
cac66f8
feat(tests): cache file.
May 25, 2024
a7c229e
feat(tests): cache file.
May 25, 2024
4ad4b9f
feat(tests): cache file.
May 25, 2024
29ec21f
feat(tests): add print for start file.
May 25, 2024
9de0989
feat(tests): fix versions for cache.
May 25, 2024
abdb933
feat(tests): fix versions for cache.
May 25, 2024
c5e1e28
feat(tests): fix cache in the ci.
May 25, 2024
87dd340
feat(tests): fix cache in the ci.
May 25, 2024
f14c91d
feat(tests): fix cache in the ci.
May 25, 2024
9811198
feat(tests): remove top import.
May 25, 2024
9c2652d
feat(tests): add new pytest-ci.ini
May 25, 2024
f5573c5
feat(tests): fix run_tests.yml
May 25, 2024
d6a36f9
feat(tests): fix run_tests.yml
May 25, 2024
5d994d4
feat(tests): fix run_tests.yml
May 25, 2024
a6f1395
feat(tests): remove pytest-ci.ini
May 25, 2024
5f31aa2
feat(tests): split requirements and requirements-dev
May 25, 2024
53fb2a5
feat(tests): revert cache
May 25, 2024
9a847f5
feat(tests): revert cache
May 25, 2024
a11e80e
feat(tests): revert pre-commit.
May 25, 2024
c1aa768
feat(tests): clean up.
May 25, 2024
f0bd17d
feat(tests): show ruff errors.
May 25, 2024
c1d49c0
feat(tests): revert requirements.
May 25, 2024
66309ee
feat(tests): fix ruff.
May 25, 2024
1be9a20
feat(tests): fix ruff.
May 25, 2024
01b671d
feat(tests): fix ruff.
May 25, 2024
26f2ffd
feat(tests): fix ruff.
May 25, 2024
b42e8bf
feat(tests): fix ruff.
May 25, 2024
c9e687e
feat(tests): cleanup.
May 25, 2024
1c65f3c
feat(tests): remove second ruff.
May 25, 2024
e06883c
feat(tests): arnav comments on intro integration
May 26, 2024
beca119
feat(tests): revert requirements.txt
May 27, 2024
481e0f1
feat(tests): remove base.py
May 27, 2024
38371f8
MIPRO optimizer updates for paper release
XenonMolecule Jun 18, 2024
8e945e7
Update test_mipro_optimizer.py
XenonMolecule Jun 18, 2024
4774de2
Ruff fixes
XenonMolecule Jun 18, 2024
ea1b328
Switch new MIPRO to MIPROv2
XenonMolecule Jun 18, 2024
34725d0
Update mipro_optimizer.py
XenonMolecule Jun 18, 2024
d1975d3
cr
jerryjliu Jun 18, 2024
0f85bc3
cr
jerryjliu Jun 18, 2024
07d8e1d
Update llamaindex.py
arnavsinghvi11 Jun 19, 2024
2ba0abb
Reduced comments/debugging statements from MIPRO dev
XenonMolecule Jun 19, 2024
95547f3
cr
jerryjliu Jun 20, 2024
70c6f21
feat(dspy): added support for managed identity in AzureOpenAI
utsavtulsyan Jun 20, 2024
00c7b65
fix(dspy): remove dummy csv file
utsavtulsyan Jun 20, 2024
c7e0116
Update README.md
okhat Jun 20, 2024
e6373de
avoid llama_index dependency breaking tests
arnavsinghvi11 Jun 21, 2024
01c8de0
Merge pull request #1170 from jerryjliu/jerry/add_llamaindex_integration
arnavsinghvi11 Jun 21, 2024
4059328
Added caching to notebooks
XenonMolecule Jun 21, 2024
f04d03c
Updated notebook docs
XenonMolecule Jun 21, 2024
1ee5479
Explained teacher settings better
XenonMolecule Jun 21, 2024
015c649
Merge pull request #1169 from stanfordnlp/mipro_v2
XenonMolecule Jun 21, 2024
de44089
Fixed notebook caches
XenonMolecule Jun 21, 2024
8909e06
Merge pull request #1185 from stanfordnlp/mipro_v2
XenonMolecule Jun 21, 2024
52ac406
Ensuring that candidate count for Gemini is always 1
Jun 21, 2024
bff6d45
Update pyproject.toml
okhat Jun 21, 2024
c65445f
Update setup.py
okhat Jun 21, 2024
e3ba07b
Fix linting issues with Ruff
Jun 22, 2024
eb6db37
Added trust_remote_code=True to hotpot_qa dataset to skip prompt for …
hmoazam Jun 22, 2024
46a3c73
update default valset
arnavsinghvi11 Jun 22, 2024
66f7f5b
Merge pull request #1194 from stanfordnlp/miprov2_patch
XenonMolecule Jun 22, 2024
d1a0d93
Merge pull request #1048 from stanfordnlp/integration-tests
okhat Jun 22, 2024
32b1060
Merge pull request #1192 from hmoazam/add_hf_param
hmoazam Jun 23, 2024
6efb0eb
Automation to release to pypi, including an intermediate deployment t…
hmoazam Jun 22, 2024
2c0305a
deleted unused file
hmoazam Jun 23, 2024
01e5066
Merge pull request #919 from hmoazam/automate-pypi-deployment
okhat Jun 23, 2024
42fb1b1
Bug - Dataset summary generation failing due to log file set to None
shubham-skr Jun 24, 2024
21d776f
Add initial template
erika-cardenas Jun 24, 2024
265a2f9
Merge pull request #1200 from shubham-skr/patch-1
XenonMolecule Jun 24, 2024
4f3e346
Update link to Weaviate recipes
erika-cardenas Jun 24, 2024
1a83af9
Add the documentation
erika-cardenas Jun 24, 2024
22bd7e9
Add link to Weaviate integration page
erika-cardenas Jun 24, 2024
d81402c
Update setup.py -- thanks Shukri!
CShorten Jun 24, 2024
0293d78
Fix Weaviate recipes link
erika-cardenas Jun 24, 2024
8d73d53
Merge pull request #1204 from erika-cardenas/add-weaviate-docs
CShorten Jun 24, 2024
d38cbf5
Update broken build for docs
krypticmouse Jun 24, 2024
14f1674
update links
krypticmouse Jun 24, 2024
7e1af2b
Use comments to mark location to replace instead of placeholders
hmoazam Jun 25, 2024
2874855
fix spaces
hmoazam Jun 25, 2024
3ade209
fix typo
hmoazam Jun 25, 2024
189f3e4
Updated to actual dspy-ai package post testing
hmoazam Jun 25, 2024
57a69b8
Merge pull request #1205 from hmoazam/fix/pkg-name-and-version
arnavsinghvi11 Jun 25, 2024
5ceb906
Update README.md
okhat Jun 25, 2024
39cda49
Merge pull request #1182 from utsavtulsyan/feature/azureopenai-manage…
arnavsinghvi11 Jun 25, 2024
c6219ca
Merge pull request #1188 from marshmellow77/fix-vertexai-gemini-candi…
arnavsinghvi11 Jun 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(tests): change it to ci way -_-
  • Loading branch information
Amir Mehr committed May 25, 2024
commit d96c0ce78d34f171d42253bb76bf809c308b825d
15 changes: 7 additions & 8 deletions tests_integration/test_intro.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from typing import Any

import dspy
from tests_integration.base import BaseIntegrationTestWithCache


Expand All @@ -9,13 +10,13 @@ def test_dspy_workflow(self) -> None:

dev_example, dev_set, training_set = self.assert_dataset_loading()

self.assert_basic_qa(dev_example, dspy)
self.assert_basic_qa(dev_example)

self.assert_retrieval(dev_example, dspy)
self.assert_retrieval(dev_example)

self.assert_compilation(dev_set, dspy, training_set)
self.assert_compilation(dev_set, training_set)

def assert_compilation(self, devset, dspy, trainset) -> None:
def assert_compilation(self, devset, trainset) -> None:
class GenerateAnswer(dspy.Signature):
"""Answer questions with short factoid answers."""

Expand Down Expand Up @@ -169,7 +170,7 @@ def validate_context_and_answer_and_hops(example, pred, trace=None):
)
assert compiled_baleen("How many storeys are in the castle that David Gregory inherited?") is not None

def assert_retrieval(self, dev_example, dspy) -> None:
def assert_retrieval(self, dev_example) -> None:
retrieve = dspy.Retrieve(k=3)
top_k_passages = retrieve(dev_example.question).passages

Expand All @@ -195,7 +196,7 @@ def assert_retrieval(self, dev_example, dspy) -> None:
retrieved_value = retrieve("When was the first FIFA World Cup held?").passages[0]
assert retrieved_value[:30] == "History of the FIFA World Cup "

def assert_basic_qa(self, dev_example, dspy) -> None:
def assert_basic_qa(self, dev_example) -> None:
class BasicQA(dspy.Signature):
"""Answer questions with short factoid answers."""

Expand Down Expand Up @@ -268,8 +269,6 @@ def assert_dataset_loading(self) -> tuple:
return dev_example, devset, trainset

def setup_dspy(self) -> Any:
import dspy

turbo = dspy.OpenAI(model="gpt-3.5-turbo")
colbertv2_wiki17_abstracts = dspy.ColBERTv2(url="http:https://20.102.90.50:2017/wiki17_abstracts")
dspy.settings.configure(lm=turbo, rm=colbertv2_wiki17_abstracts)
Expand Down