llama.cpp : add documentation about rope_freq_base and scale values #3401
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, setting
rope_freq_base
to10000
andrope_freq_scale
to1
inllama_context_params
would cause llama.cpp to use model's default value. Now, to use the model default values, these parameters must be set to zero.Setting
rope_freq_base
to10000
will cause problems with models trained on a different value, such as CodeLlama-7B. Therefore, downstream users should be careful to update the default values of these parameters.When using
llama_context_default_params()
to initializellama_context_params
without changing these parameters, no further action is required.llama_context_default_params()
will correctly set the value of these parameters to zero.