gguf : add tokenizer.chat_template documentation (ggerganov#616)

balisujohn · Nov 19, 2023 · 57c468b · 57c468b
1 parent 992439c
commit 57c468b
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/docs/gguf.md b/docs/gguf.md
@@ -512,6 +512,7 @@ Hugging Face maintains their own `tokenizers` library that supports a wide varie
 Other tokenizers may be used, but are not necessarily standardized. They may be executor-specific. They will be documented here as they are discovered/further developed.
 
 - `tokenizer.rwkv.world: string`: a RWKV World tokenizer, like [this](https://github.com/BlinkDL/ChatRWKV/blob/main/tokenizer/rwkv_vocab_v20230424.txt). This text file should be included verbatim.
+- `tokenizer.chat_template : string`: a Jinja template that specifies the input format expected by the model. For more details see: <https://huggingface.co/docs/transformers/main/en/chat_templating>
 
 ### Computation graph