Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix and improve YaRN #66

Merged
merged 2 commits into from
Nov 8, 2023
Merged

Bug fix and improve YaRN #66

merged 2 commits into from
Nov 8, 2023

Conversation

honglu2875
Copy link
Contributor

  • Some bug fixes
  • Now ModelConfig has an optional yarn_config. The model uses yarn if this is not None. Nothing should break for the original json config. And to add a yarn_config we only need to add a nested dict into the json file (see the new test_yarn.json in config folder). Also wrote a unit test to make sure the config works.

@loubbrad loubbrad self-requested a review November 8, 2023 12:55
@loubbrad loubbrad merged commit 540b3b3 into EleutherAI:main Nov 8, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants