Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Conversion Utilities #1124

Merged
merged 24 commits into from
Feb 8, 2024
Merged

Improve Conversion Utilities #1124

merged 24 commits into from
Feb 8, 2024

Conversation

haileyschoelkopf
Copy link
Contributor

@haileyschoelkopf haileyschoelkopf commented Jan 17, 2024

[Draft for now]

This PR seeks to improve the state of conversion / checkpoint import+export tools in GPT-NeoX.

Intended changes are:
NeoX to HF:

  • Unify the previously-separate PipelineModule and Sequential conversion scripts
  • Address some NeoX-to-HF QoL features (manually override checkpoint's save precision, allow to skip tokenizer upload (if one is using Tiktoken, say)
  • Allow for Mistral + Llama models, and GQA in these

Meta Llama / Mistral to NeoX:

  • enable + test GQA
  • Add Llama 2 loading instructions

Tests:

  • convert_module_to_hf.py and convert_neox_to_hf.py output precisely the same files, even under tensor parallelism (NeoX)
  • convert_sequential_to_hf.py and convert_neox_to_hf.py output precisely the same files, even under tensor parallelism (NeoX)
  • Llama model conversion functions, in the absence of GQA
  • Llama model conversion functions, with GQA

@haileyschoelkopf haileyschoelkopf marked this pull request as ready for review February 6, 2024 01:40
Quentin-Anthony
Quentin-Anthony previously approved these changes Feb 8, 2024
Copy link
Member

@Quentin-Anthony Quentin-Anthony left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All comments addressed over DM. Great work!

@Quentin-Anthony Quentin-Anthony merged commit f7373f8 into main Feb 8, 2024
2 checks passed
@Quentin-Anthony Quentin-Anthony deleted the update-conversion-utils branch February 8, 2024 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert HF format or raw weights of Llama2 to NEOX format Add Instructions for Loading Llama2 Models
2 participants