中文 | English
屏蔽不同大模型API的差异，统一用openai api标准格式使用大模型; 配置化管理不同大模型调用参数，让你在使用大模型的时候只需关注 api-key 和 messages
- 支持多种大模型，当前已支持
- openai
- azure open ai
- claude-api 【api申请在等待列表，暂未测试】
- claude-web (将web端功能封装成openai api)
- 智谱ai
- bingchat
- 百度文心一言
- 讯飞星火
- ...
- 支持stream方式调用
- 支持open ai的第三方代理服务，比如openai-sb等
- 支持在线更新配置
https://0.0.0.0:8090/（这个前端页面和交互完全是用gpt写的 哈哈） - 支持负载均衡，一个key可轮训/随机/并行等访问多个模型
-
git clone 拉取项目代码
-
cp model-config.template model-config.json并按需修改配置文件model-config.json{ "token": "f2b7295fc440db7f", "type": "azure", "config": { "api_base": "https://xxxx.openai.azure.com/", "deployment_id": "xxxx", "api_version": "2023-05-15", "api_key": "xxxx", "temperature": 0.8 } } -
本地化部署直接
pip install -r requirements.txt后，运行python open-api.py, docker部署在目录下执行docker compose up -d -
有了api-base: localhost:8090 和 api-key:f2b7295fc440db7f 可以使用了，下边列举了几种使用 `
curl https://localhost:8090/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer f2b7295fc440db7f" \
-d '{
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Hello!"
}
]
}'
import openai
openai.api_key = "f2b7295fc440db7f"
openai.api_base = "https://localhost:8090/v1"
completion = openai.ChatCompletion.create(
model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello world"}])
print(completion.choices[0].message.content)
[
{
"token": "f2b7295fc440db7f",
"type": "azure",
"config": {
"api_base": "https://xxxx.openai.azure.com/",
"deployment_id": "gpt-35-turbo",
"api_version": "2023-05-15",
"api_key": "xxxxxx",
"temperature": 0.8
}
},
{
"token": "GxqT3BlbkFJj",
"type": "openai",
"config": {
"api_base": "https://api.openai.com/v1/",
"api_key": "sk-xxxxxx",
"model": "gpt-3.5-turbo"
}
},
{
"token": "sb-ede1529390cc",
"type": "proxy",
"config": {
"api_base": "https://api.openai-sb.com/v1/",
"api_key": "sb-xxxxxx",
"model": "gpt-3.5-turbo"
}
},
{
"token": "c115c8f5082",
"type": "claude-web",
"config": {
"cookie": "xxxxxx",
"proxies": {
"https": "https://localhost:7890"
},
"conversation_id": "xxxxxx",
"prompt": "The information in [] is the context of the conversation. Please ignore the JSON format of the context during the conversation and answer the user's latest conversation: {newMessage} \n {history}",
"single_conversation": true
}
},
{
"token": "7c7aa4a3549f5",
"type": "zhipu-api",
"config": {
"api_key": "xxxxxx",
"model": "chatglm_lite",
"temperature": 0.8,
"top_p": 0.7
}
}
]