System message not supported for Anthropic (anthropic.BadRequestError)

[fronx]

Using the system role with the Anthropic backend results in the following error:

anthropic.BadRequestError: Error code: 400 - {
    'type': 'error',
    'error': {
        'type': 'invalid_request_error',
        'message': 'messages: Unexpected role "system". The Messages API accepts a top-level `system` parameter, not "system" as an input message role.'
    }
}

Is there a known workaround? I've tried using a user message instead, but occasionally that causes confusion for Haiku.

[fronx]

Here is how Anthropic system messages are supposed to be provided (source: https://docs.anthropic.com/claude/docs/system-prompts):

client = anthropic.Client(api_key="YOUR_API_KEY")

response = client.messages.create(
    model="claude-2.1",
    system="Respond only in Spanish.", # <-- system prompt
    messages=[
        {"role": "user", "content": "Hello, Claude!"} # <-- user prompt
    ]
)

And here is what the current sglang implementation does:

# …
        if s.messages_:
            messages = s.messages_
        else:
            messages = [{"role": "user", "content": s.text_}]

        ret = anthropic.Anthropic().messages.create(
            model=self.model_name,
            messages=messages,
            **sampling_params.to_anthropic_kwargs(),
        )
# …

So there is currently no way of converting a system message into a system parameter.

[fronx]

Would the following be an acceptable fix?

# …
        if s.messages_:
            messages = s.messages_
        else:
            messages = [{"role": "user", "content": s.text_}]
        
        if messages and messages[0]["role"] == "system":
            system = messages.pop(0)["content"]
        else:
            system = ""

        ret = anthropic.Anthropic().messages.create(
            model=self.model_name,
            system=system,
            messages=messages,
            **sampling_params.to_anthropic_kwargs(),
        )
# …

If I can figure out how to set things up on my machine so I can run tests for this repo, I could contribute a fix.