Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add chat templating for HF models #1287

Closed
wants to merge 31 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
3824828
first stab at wrap_chat_template
daniel-furman Jan 7, 2024
a784417
first stab at wrap_chat_template, strip error fix
daniel-furman Jan 7, 2024
53c68db
first stab at wrap_chat_template, rfind continuation fix
daniel-furman Jan 7, 2024
3e27f9d
first stab at wrap_chat_template, formatting in function
daniel-furman Jan 7, 2024
87dff8b
first stab at wrap_chat_template, print statements in loglikelihood f…
daniel-furman Jan 7, 2024
5c4d9c7
first stab at wrap_chat_template, remove system for now
daniel-furman Jan 7, 2024
e689727
first stab at wrap_chat_template, remove special chars from continuation
daniel-furman Jan 10, 2024
337c084
first stab at wrap_chat_template, remove special chars tab indenting …
daniel-furman Jan 10, 2024
6c68fd1
Merge branch 'EleutherAI:main' into main
daniel-furman Jan 10, 2024
34b32f7
first stab at wrap_chat_template, various
daniel-furman Jan 10, 2024
59e3b17
first stab at wrap_chat_template, various
daniel-furman Jan 10, 2024
7191904
first stab at wrap_chat_template, arc conversation test
daniel-furman Jan 10, 2024
9949e4f
first stab at wrap_chat_template, arc conversation test
daniel-furman Jan 10, 2024
2d3c835
first stab at wrap_chat_template, remove arc experiment
daniel-furman Jan 10, 2024
49f43f9
first stab at wrap_chat_template, various
daniel-furman Jan 10, 2024
021232b
llama test
daniel-furman Jan 11, 2024
b6c75ed
llama test
daniel-furman Jan 11, 2024
047dde8
llama test
daniel-furman Jan 11, 2024
c38b9d2
llama test
daniel-furman Jan 11, 2024
1ea8470
llama test
daniel-furman Jan 11, 2024
2e27053
llama test
daniel-furman Jan 11, 2024
43dee06
llama test
daniel-furman Jan 13, 2024
39a11d0
llama test
daniel-furman Jan 13, 2024
bbcdffb
remove system
daniel-furman Jan 13, 2024
2b40017
Merge branch 'main' into add-chat-templating
haileyschoelkopf Jan 15, 2024
c47de8b
update Instance.args setter
haileyschoelkopf Jan 15, 2024
6ca8ab1
clean up wrap_chat_template + add TODOs
haileyschoelkopf Jan 15, 2024
b8bda47
Merge branch 'main' into add-chat-templating
haileyschoelkopf Jan 15, 2024
68c30aa
push most recent code
haileyschoelkopf Jan 16, 2024
37db34c
Update lm_eval/models/huggingface.py
haileyschoelkopf Feb 27, 2024
495d50b
Update lm_eval/models/huggingface.py
haileyschoelkopf Feb 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
first stab at wrap_chat_template, remove special chars tab indenting …
…style fix
  • Loading branch information
daniel-furman committed Jan 10, 2024
commit 337c084b1b8be6c37ed4322c4822b1c68c2021dd
8 changes: 5 additions & 3 deletions lm_eval/models/huggingface.py
Original file line number Diff line number Diff line change
Expand Up @@ -684,11 +684,13 @@ def tok_wrap_chat_template(self, requests: List[Instance]) -> List[Instance]:
context = single_tokenized_conversation[:rfind_continuation]
continuation = single_tokenized_conversation[rfind_continuation:]
# remove special chars from continuation
continuation = self.tokenizer.decode(self.tokenizer.encode(continuation), skip_special_tokens=True)
continuation = self.tokenizer.decode(
self.tokenizer.encode(continuation), skip_special_tokens=True
)
req.args = (context, continuation)
new_reqs.append(req)
return new_reqs

return new_reqs


def _model_call(self, inps, attn_mask=None, labels=None):
Expand Down