Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LLM router basic template #203

Closed
wants to merge 1,502 commits into from
Closed

LLM router basic template #203

Changes from 1 commit
Commits
Show all changes
1502 commits
Select commit Hold shift + click to select a range
ea5e08c
Merge pull request #215 from anyscale/stable-diffusion-pretraining-im…
marwan116 May 16, 2024
ac68158
Merge branch 'main' into sumanthrh/function-calling
SumanthRH May 17, 2024
4eb3864
cleaned up code; added type hints and addressed review comments
SumanthRH May 17, 2024
a0aab93
minor comments
SumanthRH May 17, 2024
5129ab0
minor docstring edits
SumanthRH May 17, 2024
e83a44d
revamp preprocessing and function extraction
SumanthRH May 19, 2024
71b12bd
Apply suggestions from code review
SumanthRH May 20, 2024
cec98ba
revamp eval utils and separate openai and anyscale postprocessing
SumanthRH May 20, 2024
f56287c
added a response parser dataclass, refactored eval utils
SumanthRH May 20, 2024
835b2c7
fixed response parsing issues with tool response id
SumanthRH May 20, 2024
04a29b0
switch to simple id for function calls
SumanthRH May 20, 2024
747f1b8
remove call id in training; refactor function extraction
SumanthRH May 21, 2024
4e6e5ac
remove testing code in preprocessing
SumanthRH May 21, 2024
d1586a4
switched to colorama for pprinting
SumanthRH May 21, 2024
024adb0
minor fixes
SumanthRH May 21, 2024
050900f
minor fixes
SumanthRH May 21, 2024
52cd25f
convert pprinted string to html for display in markdown
SumanthRH May 21, 2024
35abba8
add print utils
SumanthRH May 21, 2024
e4c72b6
Updated command to non-jupyter cell
shawnanyscale May 22, 2024
dd1b06b
Merge pull request #219 from anyscale/sp/polish-rayllm
shawnanyscale May 22, 2024
51fceaf
better error handling; improved check_match logic; added base model e…
SumanthRH May 23, 2024
f080802
added final base model results and figures
SumanthRH May 23, 2024
93ba8c8
add figures
SumanthRH May 23, 2024
ed8b629
move fwd and model tag into the yamls
kouroshHakha May 23, 2024
fe1ac56
wip
kouroshHakha May 23, 2024
476f205
wip
kouroshHakha May 23, 2024
db4f7b0
safe dump
kouroshHakha May 23, 2024
8eb9c9a
wip
kouroshHakha May 23, 2024
4e05acf
Merge pull request #222 from anyscale/kh/hot-fix-ft-template-model-tag
kouroshHakha May 23, 2024
70a66e5
copy over changes from git push --set-upstream origin copykh/hot-fix-…
ArturNiederfahrenhorst May 23, 2024
17e322e
Merge pull request #223 from anyscale/copykh/hot-fix-ft-template-mode…
ArturNiederfahrenhorst May 23, 2024
a28d289
Merge branch 'main' into sumanthrh/function-calling
SumanthRH May 23, 2024
8c9dcf3
Merge pull request #214 from anyscale/sumanthrh/function-calling
SumanthRH May 24, 2024
e45bbce
fix hard-coded paths
angelinalg May 24, 2024
4bfd6a8
Update CODEOWNERS
angelinalg May 28, 2024
897f616
Merge pull request #226 from anyscale/angelinalg-patch-1
angelinalg May 28, 2024
0544bf4
Added webhook registry
shawnanyscale May 28, 2024
44e0f90
Added registry setup
shawnanyscale May 28, 2024
19fc4d0
initial
ArturNiederfahrenhorst May 16, 2024
0b35022
fix readme
ArturNiederfahrenhorst May 16, 2024
420076e
readme changes
ArturNiederfahrenhorst May 17, 2024
9b9c6dd
lint
ArturNiederfahrenhorst May 16, 2024
420ad49
wip
ArturNiederfahrenhorst May 23, 2024
7566f3f
Add adapter etc
ArturNiederfahrenhorst May 28, 2024
f10b5d7
Fill in REAMDME
ArturNiederfahrenhorst May 28, 2024
3ec5f81
make everything a cookbook
ArturNiederfahrenhorst May 28, 2024
bf179bc
lint
ArturNiederfahrenhorst May 28, 2024
22a06d7
delete adapter and fix README
ArturNiederfahrenhorst May 28, 2024
4b6b1bd
fix notebook
ArturNiederfahrenhorst May 28, 2024
b51eb80
revert templates/batch-llm/README.md chnges
ArturNiederfahrenhorst May 29, 2024
01a9c0b
[Triton] Add Triton template to run Stable Diffusion on Anysacle Plat…
GeneDer May 30, 2024
f94d964
Merge branch 'main' into genesu/triton-on-anyscale-services
GeneDer May 30, 2024
07832ba
try to fix it
GeneDer May 30, 2024
8405e81
Fix Serve docs links
emmyscode May 30, 2024
51630ea
Merge pull request #229 from anyscale/fix-docs-links
emmyscode May 30, 2024
8a7789c
add link in first paragraph, fix link to intro to jobs
angelinalg May 30, 2024
b25f9b3
Updated
shawnanyscale May 30, 2024
ac83669
fix link to first page of examples directory
angelinalg May 31, 2024
c483f7e
Merge branch 'main' into fix-links
angelinalg May 31, 2024
48b2ed3
Merge pull request #225 from anyscale/fix-links
emmyscode May 31, 2024
d597a81
remove README.md
GeneDer May 31, 2024
ee4a991
Checkpoint
shomilj May 31, 2024
35a3e0e
Fix Emmy
shomilj May 31, 2024
8f9793c
Remove emmy
shomilj May 31, 2024
9f1b146
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst May 31, 2024
793ce85
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst May 31, 2024
7cd0095
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst May 31, 2024
72a13ba
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst May 31, 2024
459e59f
kourosh's comments
ArturNiederfahrenhorst May 31, 2024
82a4b80
Merge branch 'main' into continuedfttemplate
ArturNiederfahrenhorst May 31, 2024
9dbc8c8
Merge pull request #231 from anyscale/shomilj/fix-codeowner
shomilj Jun 1, 2024
3eae29b
address comments
GeneDer Jun 2, 2024
55ca07f
kourosh's comments
ArturNiederfahrenhorst Jun 2, 2024
0bd3e94
Merge branch 'continuedfttemplate' of github.com:anyscale/templates i…
ArturNiederfahrenhorst Jun 2, 2024
2c2f7fe
Add yaml part
ArturNiederfahrenhorst Jun 2, 2024
79c8b32
fix images
ArturNiederfahrenhorst Jun 2, 2024
0c105ba
Merge branch 'main' into continuedfttemplate
ArturNiederfahrenhorst Jun 2, 2024
159ac7a
fix pics
ArturNiederfahrenhorst Jun 2, 2024
5269c51
Merge branch 'continuedfttemplate' of github.com:anyscale/templates i…
ArturNiederfahrenhorst Jun 2, 2024
bb27e10
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
14cacb3
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
22904bd
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
304ae86
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
974e874
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
1f81b69
add download pic
ArturNiederfahrenhorst Jun 2, 2024
5e45c24
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
5002940
Update templates/fine-tune-llm_v2/cookbooks/continue_from_checkpoint/…
ArturNiederfahrenhorst Jun 2, 2024
2b74fa4
add continue ft png
ArturNiederfahrenhorst Jun 2, 2024
e2b4201
add folder
ArturNiederfahrenhorst Jun 2, 2024
efc9916
Update templates/fine-tune-llm_v2/README.ipynb
ArturNiederfahrenhorst Jun 3, 2024
c41b506
Update templates/fine-tune-llm_v2/README.md
ArturNiederfahrenhorst Jun 3, 2024
f82e96c
kourosh's last comment
ArturNiederfahrenhorst Jun 3, 2024
823b90b
fix step 3 title
GeneDer Jun 3, 2024
ac298fa
[llm-ft-template] Add cookbook for continued fine-tuning
kouroshHakha Jun 3, 2024
ebaac0b
initial
ArturNiederfahrenhorst Jun 3, 2024
d70fe60
fix fp config and readme
ArturNiederfahrenhorst Jun 3, 2024
bbd5040
fix headline
ArturNiederfahrenhorst Jun 3, 2024
ac3c75b
address more comments and run through vale
GeneDer Jun 3, 2024
f0fada6
fix the wrong config files
GeneDer Jun 3, 2024
53d47dd
adding function-calling
kouroshHakha Jun 4, 2024
774ffc8
Add proper runtimes
ArturNiederfahrenhorst Jun 4, 2024
fbd6432
Kourosh's research comment
ArturNiederfahrenhorst Jun 4, 2024
c23ee66
Updated naming
shawnanyscale Jun 4, 2024
7bedc7f
Merge pull request #233 from anyscale/sp/update-name
shawnanyscale Jun 4, 2024
0ee9dac
cleaned up some path stuff
kouroshHakha Jun 4, 2024
6cf2e1e
generated README.md files
kouroshHakha Jun 4, 2024
d5f5471
pre-commit fixes
kouroshHakha Jun 4, 2024
df6321c
fixed image paths
kouroshHakha Jun 4, 2024
7774110
readme
kouroshHakha Jun 4, 2024
8df0a6c
update
kouroshHakha Jun 4, 2024
7ec190a
address more comments
GeneDer Jun 4, 2024
4fe95c2
Update templates/fine-tune-llm_v2/README.ipynb
kouroshHakha Jun 5, 2024
0823c3c
Update templates/fine-tune-llm_v2/README.ipynb
kouroshHakha Jun 5, 2024
5bb0249
Update templates/fine-tune-llm_v2/end-to-end-examples/fine-tune-funct…
kouroshHakha Jun 5, 2024
bd7555f
Update templates/fine-tune-llm_v2/end-to-end-examples/fine-tune-funct…
kouroshHakha Jun 5, 2024
c18419d
fixes
kouroshHakha Jun 5, 2024
5901fa0
Merge branch 'main' into kh/restructure-ft-template
shawnanyscale Jun 5, 2024
bb3a4e3
[ft-template] Add function calling to the Fine-tuning template + some…
kouroshHakha Jun 5, 2024
2b80846
Merge branch 'main' into genesu/triton-on-anyscale-services
GeneDer Jun 5, 2024
99b1d55
add cookbook to bring your own data
SumanthRH Jun 6, 2024
b0bf1bd
add readme
SumanthRH Jun 6, 2024
e2392d5
add cookbook for hyperparameters
SumanthRH Jun 6, 2024
20ce33a
add readme
SumanthRH Jun 6, 2024
3caa18f
Merge branch 'main' into loravsfullparam
ArturNiederfahrenhorst Jun 6, 2024
e07482f
Merge branch 'main' into sp/add-webhook
shawnanyscale Jun 6, 2024
172b194
Merge pull request #227 from anyscale/sp/add-webhook
shawnanyscale Jun 6, 2024
0cef6bc
minor edit
SumanthRH Jun 6, 2024
7fa7c6f
edit gcp upload link
SumanthRH Jun 6, 2024
7eab268
minor edits
SumanthRH Jun 7, 2024
1338191
minor edits
SumanthRH Jun 7, 2024
b14095d
fix grammar
GeneDer Jun 7, 2024
938a2eb
Merge branch 'main' into genesu/triton-on-anyscale-services
GeneDer Jun 7, 2024
3977f4a
Merge pull request #228 from anyscale/genesu/triton-on-anyscale-services
GeneDer Jun 7, 2024
ae839bb
add optimum installation
scottjlee Jun 7, 2024
1b064b7
use optimum==1.19.2
scottjlee Jun 7, 2024
fb1cf3f
Merge branch 'main' into release-text-embeddings-0605
scottjlee Jun 7, 2024
a057af7
[ft-template] Added cook book for optimizing cost and performance
kouroshHakha Jun 8, 2024
8e49e53
wip
kouroshHakha Jun 8, 2024
83f6600
Merge branch 'main' into sumanthrh/ft-template-hyperparams
SumanthRH Jun 8, 2024
64631c9
Merge remote-tracking branch 'origin' into sumanthrh/ft-template-data
SumanthRH Jun 8, 2024
e28479d
address review comments
SumanthRH Jun 8, 2024
c24a4b4
minor edits
SumanthRH Jun 8, 2024
7770693
minor edits
SumanthRH Jun 8, 2024
8f4a1bd
Apply suggestions from code review
SumanthRH Jun 10, 2024
c48fc4c
initial commit for llm-router
anmscale May 4, 2024
5ea7ad0
adding data formatting
anmscale May 6, 2024
e97b93d
adding offline evaluation
anmscale May 7, 2024
0dd9b38
basic step-1 and step-3
anmscale May 7, 2024
a28b77b
delete unused file
anmscale May 7, 2024
ca87095
clean up eval metrics
anmscale May 7, 2024
5adefad
address review comments
SumanthRH Jun 10, 2024
0e699de
minor edits to wording
SumanthRH Jun 10, 2024
64b76ec
[ft-template] Add cookbook to bring your own data
kouroshHakha Jun 11, 2024
78651ea
address review comments
SumanthRH Jun 11, 2024
7384aec
minor fix
SumanthRH Jun 11, 2024
1a3e46d
Merge remote-tracking branch 'origin' into sumanthrh/ft-template-hype…
SumanthRH Jun 11, 2024
02ee186
minor fix
SumanthRH Jun 11, 2024
08f0972
Kourosh's comments
ArturNiederfahrenhorst Jun 11, 2024
8f9b5fe
fix paths
ArturNiederfahrenhorst Jun 11, 2024
9b668ce
merge main
ArturNiederfahrenhorst Jun 11, 2024
e75f46a
nit
ArturNiederfahrenhorst Jun 11, 2024
7365668
added e2e llm workflows template and exception to README.md generation
GokuMohandas Jun 11, 2024
38d1e92
fixed end of files
GokuMohandas Jun 11, 2024
d05717b
added e2e llm workflow exception to ci readme generation
GokuMohandas Jun 11, 2024
e7828ec
added e2e llm workflow exception to ci readme generation
GokuMohandas Jun 11, 2024
c3df134
made executable
GokuMohandas Jun 11, 2024
93506a1
e2e llm guide ci fixed
GokuMohandas Jun 11, 2024
ba88102
precommit checks
GokuMohandas Jun 11, 2024
e109371
precommit checks
GokuMohandas Jun 11, 2024
4570792
added configs for e2e llm workflows
GokuMohandas Jun 11, 2024
6fd850f
Merge pull request #242 from anyscale/e2e-llm
angelinalg Jun 11, 2024
d84fc92
added note on waiting time
GokuMohandas Jun 11, 2024
8710c29
Merge branch 'main' into e2e-llm
GokuMohandas Jun 11, 2024
b23c84a
Merge pull request #243 from anyscale/e2e-llm
angelinalg Jun 11, 2024
6fc7508
added note to terminate the service
GokuMohandas Jun 11, 2024
7e567b1
Merge branch 'e2e-llm' of https://github.com/anyscale/templates into …
GokuMohandas Jun 11, 2024
842160d
Merge branch 'main' into e2e-llm
GokuMohandas Jun 11, 2024
8d001cb
Merge pull request #244 from anyscale/e2e-llm
akshay-anyscale Jun 11, 2024
73ab27b
add toc
SumanthRH Jun 12, 2024
e147a80
minor fix
SumanthRH Jun 12, 2024
4672846
Merge branch 'main' into sumanthrh/ft-template-hyperparams
SumanthRH Jun 12, 2024
024b775
[ft-template] Add cookbook for modifying hyperparameters
kouroshHakha Jun 12, 2024
f18cd2b
Wip
kouroshHakha Jun 12, 2024
b907af9
wip
kouroshHakha Jun 12, 2024
d9ee49d
Merge branch 'main' into kh/templates-cost-throughput
kouroshHakha Jun 12, 2024
764e3cd
wip
kouroshHakha Jun 12, 2024
b92c04a
wip
kouroshHakha Jun 12, 2024
fcca709
wip
kouroshHakha Jun 12, 2024
17678b0
[ft-template] Added cookbook for optimizing cost and performance
kouroshHakha Jun 12, 2024
84482a8
Merge branch 'main' into loravsfullparam
kouroshHakha Jun 12, 2024
6879a70
wip
kouroshHakha Jun 12, 2024
494e9e2
[ft-template] Add llm-forge cookbook: full-parameter vs LoRA
kouroshHakha Jun 13, 2024
e0aa32b
modifications
anmscale Jun 13, 2024
f4e8f10
added data write solution for hosted anyscale
GokuMohandas Jun 13, 2024
3a950e2
Update templates/e2e-llm-workflows/README.ipynb
matthewdeng Jun 13, 2024
362c12f
Merge branch 'main' into e2e-llm
matthewdeng Jun 13, 2024
15d83d3
Merge pull request #246 from anyscale/e2e-llm
GokuMohandas Jun 13, 2024
361c4a8
pinned version for s3fs
GokuMohandas Jun 13, 2024
cc30e27
Merge branch 'main' into e2e-llm
GokuMohandas Jun 13, 2024
cda29de
removed cell block nums for e2e llm workflow template
GokuMohandas Jun 13, 2024
69ea7af
Merge branch 'e2e-llm' of https://github.com/anyscale/templates into …
GokuMohandas Jun 13, 2024
53d6c97
Merge pull request #247 from anyscale/e2e-llm
GokuMohandas Jun 13, 2024
3450faf
using lora adapter for eval
GokuMohandas Jun 13, 2024
5ecb265
Merge branch 'main' into e2e-llm
GokuMohandas Jun 13, 2024
084a1ce
Merge pull request #248 from anyscale/e2e-llm
GokuMohandas Jun 14, 2024
608b4a7
Merge branch 'main' into release-text-embeddings-0605
scottjlee Jun 17, 2024
7124707
Merge pull request #238 from anyscale/release-text-embeddings-0605
scottjlee Jun 17, 2024
1e7cdcc
latest changes
anmscale Jun 18, 2024
489c2ab
fix bug
anmscale Jun 18, 2024
6fb5815
Merge branch 'main' of github.com:anyscale/templates into main
anmscale Jun 22, 2024
2d858dc
initial commit for llm-router
anmscale May 4, 2024
1e81149
adding data formatting
anmscale May 6, 2024
dd91c8c
adding offline evaluation
anmscale May 7, 2024
3ed8a6c
basic step-1 and step-3
anmscale May 7, 2024
35cdba4
delete unused file
anmscale May 7, 2024
50cae77
clean up eval metrics
anmscale May 7, 2024
87aaa14
modifications
anmscale Jun 13, 2024
bfa1b28
latest changes
anmscale Jun 18, 2024
e548105
fix bug
anmscale Jun 18, 2024
ae9ce6c
more changes
anmscale Jun 21, 2024
b589cba
working job submission
anmscale Jun 22, 2024
ad924ae
updating notebook
anmscale Jun 22, 2024
4798b15
Merge branch 'anm/llm-router' of github.com:anyscale/templates into a…
anmscale Jun 22, 2024
c16e234
minor modifications
anmscale Jun 22, 2024
234a52e
remove wandb token
anmscale Jun 22, 2024
750a2af
new flowchart, configs
anmscale Jun 24, 2024
5375669
fix trailing space
anmscale Jun 24, 2024
aeb7f3a
adding autogen README.md
anmscale Jun 24, 2024
a16661b
remove datafiles
anmscale Jun 24, 2024
e684cd5
remove additional space
anmscale Jun 24, 2024
1791f98
add end line
anmscale Jun 24, 2024
0fd2bcc
Merge branch 'main' into anm/llm-router
GokuMohandas Jun 25, 2024
ed80144
add md files
anmscale Jun 25, 2024
3fdd882
Merge branch 'anm/llm-router' of github.com:anyscale/templates into a…
anmscale Jun 25, 2024
d19563c
feature complete
anmscale Jun 25, 2024
0df9166
remove md files
anmscale Jun 26, 2024
bbe31a4
md modifs
anmscale Jun 26, 2024
078effc
blog post changes
anmscale Jun 27, 2024
3ed3003
running e2e
anmscale Jun 27, 2024
e0d2161
change instance types
anmscale Jun 27, 2024
2ad2f7a
Merged main into your-branch-name and accepted all changes from main
anmscale Jul 8, 2024
261ac02
latest changes
anmscale Jul 8, 2024
62d6d47
update configs
anmscale Jul 8, 2024
3e22b28
cleanup GPU mem before batch eval
anmscale Jul 8, 2024
6715890
move llm-router to ft e2e examples
anmscale Jul 8, 2024
d3be03a
edit configs
anmscale Jul 8, 2024
d29f2ad
remove configs
anmscale Jul 8, 2024
55ee2e2
put llm-router specific configs
anmscale Jul 8, 2024
656c650
moving to a separate template
anmscale Jul 8, 2024
da71a45
update to recent llmforge image
anmscale Jul 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merged main into your-branch-name and accepted all changes from main
  • Loading branch information
@anmscale
anmscale committed Jul 8, 2024
commit 2ad2f7a474ae679d4863dd976d62f6c67bc0aed0

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.