Enhancement of chat template handling #4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces several enhancements. Specifically, it focuses on the
wrap_chat_template
andlog_likelihood
methods within thehuggingface.py
module.Context of the Changes
These changes are based on the unmerged PR (EleutherAI#1287) from the original repository. The mentioned PR contains valuable improvements that are not yet merged into the main branch. By integrating these improvements into our fork, we aim to enhance the model's interaction with chat templates and accuracy in performance evaluation.
Changes Made
Instance Class Enhancements
args
Property with Setter: A new propertyargs
with a setter has been added to theInstance
class to update the instance's arguments. This ensures that the new arguments match the old ones in length and type, enhancing the integrity of instance modifications.Huggingface Model Improvements
use_chat_template
andsystem_prompt
as optional parameters to theHFLM
class to facilitate the handling of chat templates.wrap_chat_template
method has been improved to include system prompts and better prepare the context for text generation.generation_kwargs
to remove thetemperature
parameter whendo_sample
is set toFalse
, providing a more consistent deterministic generation behavior.loglikelihood
method now utilizes the chat template functionality when enabled, with debug prints added for monitoring the prompt formatting process.Additional Notes
Impact
The proposed enhancements are expected to improve the model's flexibility and reliability when dealing with chat templates. They also aim to provide a more accurate and performant evaluation of the model's capabilities.
Please review the proposed changes and provide feedback on any further requirements or amendments needed.
I look forward to your feedback and am ready to make any necessary adjustments to prepare this PR for merging.
Thank you for considering this contribution to the project.