You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have deployed mistral7b and 8x&B models in two AWS EC2 server using vllm and fstchat , and now I want to communicate through a web interface(gradio_webserver) and i am using api_endpoints.json file to register models.
I have deployed mistral7b and 8x&B models in two AWS EC2 server using vllm and fstchat , and now I want to communicate through a web interface(gradio_webserver) and i am using api_endpoints.json file to register models.
api_endpoints.json
{
"Mistral": {
"model_name": "mistral7b",
"api_type": "openai",
"api_base": "https://10.0.0.0/v1",
"api_key": "sk-******",
"anony_only": false,
"recommended_config": {
"temperature": 0.7,
"top_p": 1.0
},
"text-arena": true,
"vision-arena": false,
}
}
error from interface:
![image](https://private-user-images.githubusercontent.com/75029215/340902944-3d877bb1-f1dc-412f-93c9-55f4892913c5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NTExMjQsIm5iZiI6MTcyNDg1MDgyNCwicGF0aCI6Ii83NTAyOTIxNS8zNDA5MDI5NDQtM2Q4NzdiYjEtZjFkYy00MTJmLTkzYzktNTVmNDg5MjkxM2M1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEzMTM0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTM4NWZkMzIwYmI0ZmY4M2EzYmYwYTdjZjA0ZWQ3Yjg1NzhhZGUwNTA3YTE1MjhiZjA3Zjg0NGU2ODRkYzg5MmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.GPRAWG62g5Fi4XQiXuzxmvcLDoqZGhw82fgfPqEenbc)
command used to launch webUI
python3 -m fastchat.serve.gradio_web_server --host 0.0.0.0 --port 8080 --controller-url "" --share --register api_endpoints.json
The text was updated successfully, but these errors were encountered: