New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SamplingParams add "spaces_between_special_tokens" argument #392

Merged

merrymercy merged 1 commit into sgl-project:main from ZX-ModelCloud:SamplingParams_add_spaces_between_special_tokens

Apr 30, 2024

Contributor

ZX-ModelCloud commented Apr 25, 2024 •

edited

Loading

Add tokenizer decode parameter spaces_between_special_tokens.

See: https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L95


SamplingParams add "spaces_between_special_tokens" argument

45b6966

Contributor

Qubitium commented Apr 26, 2024

We have tested this and this param is important to normalize fast vs non-fast hf tokenizers as to how spaces are handled.

merrymercy merged commit 183df47 into sgl-project:main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment