Skip to content

Commit

Permalink
gguf : add keys for kv sizes to spec (ggerganov#676)
Browse files Browse the repository at this point in the history
* Add keys for kv sizes to GGUF spec

* Fix types of key_length and value_length
  • Loading branch information
postmasters committed Jan 5, 2024
1 parent 3eace58 commit 9e22103
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/gguf.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,8 @@ In the following, `[llm]` is used to fill in for the name of a specific LLM arch
- `[llm].attention.clamp_kqv: float32`: Value (`C`) to clamp the values of the `Q`, `K`, and `V` tensors between (`[-C, C]`).
- `[llm].attention.layer_norm_epsilon: float32`: Layer normalization epsilon.
- `[llm].attention.layer_norm_rms_epsilon: float32`: Layer RMS normalization epsilon.
- `[llm].attention.key_length: uint32`: The optional size of a key head, $d_k$. If not specified, it will be `n_embd / n_head`.
- `[llm].attention.value_length: uint32`: The optional size of a value head, $d_v$. If not specified, it will be `n_embd / n_head`.
#### RoPE
Expand Down

0 comments on commit 9e22103

Please sign in to comment.