Google Gemma

[surak]

Google released their new open model, Gemma. https://huggingface.co/google/gemma-7b-it

But we can't use it on fastchat with sglang yet:

File "/p/haicluster/llama/FastChat/sc_venv_sglang/sglang/python/sglang/srt/managers/router/model_runner.py", line 49, in get_model_cls_by_arch_name
2024-02-21 15:21:34 | INFO | stdout |     raise ValueError(
2024-02-21 15:21:34 | INFO | stdout | ValueError: Unsupported architectures: GemmaForCausalLM. Supported list: ['QWenLMHeadModel', 'Qwen2ForCausalLM', 'LlavaLlamaForCausalLM', 'MixtralForCausalLM', 'MistralForCausalLM', 'YiVLForCausalLM', 'LlamaForCausalLM']

[nopepper]

👍 I'd also be interested in this. I don't know how hard it would be to add support, though.

[Arcmoon-Hu]

Apparently sglang doesn't support this model yet. May be you can create a branch support it.