Skip to content

Issues: mit-han-lab/qserve

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

19 Open 4 Closed
19 Open 4 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How can we reproduce Table.2 and 3 ? (PPL and zero-shot Acc)
#25 opened Jul 12, 2024 by kriskrisliu
2
[New Model Supported] MiniCPM-2B
#24 opened Jul 8, 2024 by RanchiZhao
Question about dequantization overhead
#23 opened Jul 6, 2024 by DD-DuDa
2
Circular import error
#22 opened Jul 5, 2024 by LuckyLYM
1
The outpout of given model(mit-han-lab/Llama-3-8B-QServe-g128) is mistaken
#21 opened Jul 2, 2024 by haichuan1221
1
Expected speed for llama3-70b-instruct
#18 opened Jun 4, 2024 by ethxnp
1
Is the Table.3 accuracy tested with dequantized weights, or tested on real accelerated quantized kernels?
#17 opened Jun 3, 2024 by vovoluck
1
has anyone tried to HIPify this for AMD/ROCm
#16 opened Jun 2, 2024 by ehartford
support tp
#14 opened May 24, 2024 by cyLi-Tiger
2
activation quantization
#13 opened May 24, 2024 by hanhanpp
1
Any performance comparsion with vllm?
#12 opened May 21, 2024 by MuYu-zhi
1
Llama-2-7B-QServe model doesn't give the expected output
#11 opened May 21, 2024 by MuYu-zhi
2
Question about the paper
#10 opened May 18, 2024 by jameswu2014
3
Couldn't instantiate the backend tokenizer
#8 opened May 16, 2024 by Rudin6
1
lmquant for QoQ quantization and fake-quantized model dumping
#7 opened May 15, 2024 by SimpleTheoryOfTypes
1
Would this work on consumer hardware and integrated in frameworks like llama.cpp or others?
#5 opened May 11, 2024 by Mayorc1978
3
Source code
#3 opened May 9, 2024 by jph00
2
Is 8bit supported?
#2 opened May 8, 2024 by nivibilla
3
How to Quantize CNN Layers (non MLP layers in general) using qserve?
#1 opened May 8, 2024 by satabios
1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.