Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add keys for kv sizes to GGUF spec #676

Merged
merged 2 commits into from
Jan 5, 2024
+2 −0
Merged

Add keys for kv sizes to GGUF spec #676

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d8ac0d
Add keys for kv sizes to GGUF spec
postmasters Jan 2, 2024
7e9d67c
Fix types of key_length and value_length
postmasters Jan 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add keys for kv sizes to GGUF spec
  • Loading branch information
@postmasters
postmasters committed Jan 2, 2024
commit 8d8ac0d7f3fbe9e0e19bcd890df3e9ad691d13b4
2 changes: 2 additions & 0 deletions docs/gguf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,8 @@ In the following, `[llm]` is used to fill in for the name of a specific LLM arch
- `[llm].attention.clamp_kqv: float32`: Value (`C`) to clamp the values of the `Q`, `K`, and `V` tensors between (`[-C, C]`).
- `[llm].attention.layer_norm_epsilon: float32`: Layer normalization epsilon.
- `[llm].attention.layer_norm_rms_epsilon: float32`: Layer RMS normalization epsilon.
- `[llm].attention.key_length: uint64`: The optional size of a key head, $d_k$. If not specified, it will be `n_embd / n_head`.
- `[llm].attention.value_length: uint64`: The optional size of a value head, $d_v$. If not specified, it will be `n_embd / n_head`.

#### RoPE

Expand Down