Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build2024 #1

Draft
wants to merge 104 commits into
base: main
Choose a base branch
from
Draft

Build2024 #1

wants to merge 104 commits into from

Conversation

cedricvidal
Copy link
Owner

No description provided.

@cedricvidal
Resolve default logging conf file relative to logconf.py
4780d60
@cedricvidal
RAFT FTaaS Notebook
ab80ca1
@cedricvidal
FTaaS Pin azure-storage-file-datalake==12.14.0 to avoid dependency co…
08783a0 
…nflict
@cedricvidal
Fails creating job
9127ff9
@cedricvidal
Dataset
a0185d9
@cedricvidal
Adding deployment
e5c2788
@cedricvidal
Stopping attempts to do full code first FTaaS
a990a3a
@cedricvidal
eval notebook
727c257 
eval clear outputs
@cedricvidal
Eval progress
b48472a
@cedricvidal
Format support for eval format
d24e4e6
@cedricvidal
Delete dataset files
8544fba
@cedricvidal
WIP
160240f
@cedricvidal
Using instruction field for prompt
0868f9b
@cedricvidal
RAFT Fix wrong chunk being checkpointed
ead7a37 
With 30 chunks, if the process fails at chunk 29 (0-indexed), then checkpoint saves 30//15*15 = 15 instead of 30

Moreover, no need to save checkpoint state for every chunk, it's only need when the ds is actually saved and makes the code easier to understand.
@cedricvidal
RAFT format.py Fix jsonl input type
9ff66b2
@cedricvidal
RAFT ftaas notebook splitting datasets before converting
290e9fa
@cedricvidal
ARFT format.py more logs
81ea8eb
@cedricvidal
RAFT format.py Field names can be customized
1e69ede 
because different systems have different expectations.
@cedricvidal
RAFT format.py support for eval format
24fff86
@cedricvidal
RAFT ftaas.ipynb changing column names
66c6dbd
@cedricvidal
RAFT eval.ipynb eval format
1019f0f
@cedricvidal
RAFT format.py renamed answer to gold_answer
b56e4b6 
To clarify that this is the ground truth gold answer as opposed to the answer generated by the fine tuned model
@cedricvidal
RAFT pfeval.py
4cfaf39
@cedricvidal
RAFT eval notebook works end to end
67956a2
@cedricvidal
RAFT raft.py add --checkpoint-size arg
3649c7c
@cedricvidal
RAFT raft.py fix chunk and progress logging
217f54e
@cedricvidal
RAFT raft.py saving checkpoints in a single directory
cde331c 
- makes collecting checkpoints less expensive
- makes deleting checkpoints a single line call
@cedricvidal
RAFT ftaas notebook WIP
1817873
@cedricvidal
RAFT raft.py disbale datasets progress bars when saving checkpoints
b8bbc84
@cedricvidal
RAFT raft.py add llama template
838a7c9
@cedricvidal
RAFT raft.py Final log
a31bfee
@cedricvidal
BUILD gen notebook looks ready
e0deb89
@cedricvidal
RAFT raft.py more logging at the end
c895200
@cedricvidal
RAFT eval.py retry and --workers param
38a2f64
@cedricvidal
RAFT client_utils.py stats support for completion
a7b4034
@cedricvidal
RAFT eval.py more robust
c5d631e
@cedricvidal
RAFT pfeval.py remote mode
6eab9ca
@cedricvidal
RAFT raft.py tqdm postfix
50bc7a9
@cedricvidal
RAFT eval.py logging retry stats
925ab61
@cedricvidal
RAFT eval.py fixed main thread silent fail in case of exception
074ec5a
@cedricvidal
RAFT fomrat.py 'answer' column is optional
96860f5
@cedricvidal
RAFT pfeval.py remote support + local fixes
2ad000a
@cedricvidal
RAFT pfeval_local.py
ce84d47
@cedricvidal
RAFT pfeval_local.py more checks
2b9b251
@cedricvidal
RAFT pfeval_local.py fix bug
3284737
@cedricvidal
RAFT pfeval_local.py fix other bug
5ec43bf
@cedricvidal
RAFT gen notebook
0c04254
@cedricvidal
RAFT gen notebook cleaned cells
be42647
@cedricvidal
Remove ftaas
a5c2f6c
@cedricvidal
RAFT gen notebook display ft and eval datasets
504da8e
@cedricvidal
RAFT adding promptflow flows
a3564be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@cedricvidal