-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mistral model no longer loads following PR#101 #107
Comments
johndun
changed the title
Mistral model no longer loads following CR#101
Mistral model no longer loads following PR#101
Jan 26, 2024
Thanks for pointing out this issue and the workaround. I'll take a look today. |
Ok it turns out that we should do exactly what you proposed. Mistral config does use |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
get_model_cls_by_arch_name
introduced in Dynamic model class loading PR removes the hard-coded mapping betweenMistralForCausalLM
andLlamaForCausalLM
causing issues trying to local host Mistral-7b model as of sglang version 0.1.9. I have tested that adding the following simplemodels/mistral.py
file allows hosting the mistral-7b model.The text was updated successfully, but these errors were encountered: