The question regarding local deployment of large-scale models

[jialinhappy]

Hello, I noticed that your interface can only be based on llama or gpt. If I want to deploy some local variants based on llama, such as qwen or yi, how can I use them?

[GasolSun36]

Hello, I noticed that your interface can only be based on llama or gpt. If I want to deploy some local variants based on llama, such as qwen or yi, how can I use them?

Hi,
You can refer to the official implementation code to complete this task. It should be relatively simple. We only implemented the version using llama and gpt.