-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bedrock Model API #1486
Comments
Hi, we don't currently support the Amazon Bedrock API, though we'd gladly accept a PR adding an LM class for this. Are there any sample code snippets invoking the bedrock API? in an ideal world, if the Bedrock API supported an OpenAI-like API, we could run it through the existing |
Hi @haileyschoelkopf thanks for your response. The Bedrock API can be called through the import boto3
bedrock_runtime_client = boto3.client(service_name="bedrock-runtime")
enclosed_prompt = "Human: " + prompt + "\n\nAssistant:"
body = {
"prompt": enclosed_prompt,
"max_tokens_to_sample": 200,
"temperature": 0.5,
"stop_sequences": ["\n\nHuman:"],
}
response = bedrock_runtime_client.invoke_model(
modelId="anthropic.claude-v2:1", body=json.dumps(body)
)
response_body = json.loads(response["body"].read())
completion = response_body["completion"] Docs: https://docs.aws.amazon.com/code-library/latest/ug/python_3_bedrock-runtime_code_examples.html |
Thanks @austinmw ! Would you have any bandwidth / interest in contributing this? |
Hi, does this library support the Amazon Bedrock API (for example, using Claude through Bedrock)?
The text was updated successfully, but these errors were encountered: