Completion request error for LogitBias

[henrylamb]

Hi,

So there appears to be an issue when making a request using logitBias. The below error keeps on popping up when the field is populated with any int value.

Invalid key in 'logit_bias': You. You should only be submitting non-negative integers.

Any help with the above issue would be appreciated.

Below is the completion request I've used with a fine-tuned model.

req := gogpt.CompletionRequest{
	Model:            curieModelV5,
	MaxTokens:        110, //110
	Prompt:           input,
	Stream:           false,           //below values were for V2 -- For V5
	Temperature:      0.9,             //0.65 -- 0.95
	TopP:             1,               //0.95 -- 0.95
	FrequencyPenalty: 0.5,             //0.5 -- 0.05
	PresencePenalty:  0.2,             //0.8 -- 0.15
	BestOf:           1,               //4
	LogProbs:         30,              // didn't exist -- 5
	Stop:             []string{"END"}, // didn't exist --"END"
	N:                1,
	LogitBias:        map[string]int{" You": 6}, //doesn't work in the request
}

[andrice7]

I'm not experienced enough to know for sure, but I think there is a bug in a marshaller. I'm seeing if I can fix the issue myself. I'm mostly new tho.

[vvatanabe]

@henrylamb @andrice7

The LogitBias key must be a token id string (specified by their token ID in the tokenizer) , not a word string.

incorrect: "logit_bias":{"You": 6}
correct: "logit_bias":{"1639": 6}

Please see:

• https://platform.openai.com/docs/api-reference/chat/create
• https://community.openai.com/t/logitbias-not-working-properly-for-chatcompletions-api/80883/7

[vvatanabe]

I've added comments to clarify the values of LogitBias, as they were difficult to understand. #426