Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the tokens by removing whitespace #5

Closed
TimPietrusky opened this issue Apr 4, 2023 · 1 comment · Fixed by #16
Closed

Optimize the tokens by removing whitespace #5

TimPietrusky opened this issue Apr 4, 2023 · 1 comment · Fixed by #16

Comments

@TimPietrusky
Copy link
Member

No description provided.

@pixelass
Copy link
Member

pixelass commented Apr 4, 2023

It is crucial that comments remain untouched and the minifier supports esm. We have a custom prototype which works (with flaws). The removal of whitespaces has a big impact since each white space counts as a separate token. We also need the responses to follow those rules, which requires some adjustments to our system message.

The limit is 4096 token (user + assistant) e.g. user uses 1000 tokens, then the assistant can use the remaining ~3000 tokens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants