Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q4 2024
#9006 opened Oct 1, 2024 by simon-mo
Open 4
vLLM's V2 Engine Architecture
#8779 opened Sep 24, 2024 by simon-mo
Open 7
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1,699 Open 3,096 Closed
1,699 Open 3,096 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug]: vllm mistralai--Codestral-22B-v0.1 response is truncated bug Something isn't working
#9310 opened Oct 12, 2024 by Fly-Pluche
1 task done
1
[Bug]: Process group watchdog thread terminated with exception: CUDA error: an illegal memory access was encountered bug Something isn't working
#9308 opened Oct 12, 2024 by eyuansu62
1 task done
[Bug]: latest docker build (0.6.2) got error due to VLLM_MAX_SIZE_MB bug Something isn't working
#9307 opened Oct 12, 2024 by ZJLi2013
1 task done
[Bug]: Failed to pickle inputs of failed execution: CUDA error: an illegal memory access was encountered bug Something isn't working
#9306 opened Oct 12, 2024 by Clint-chan
1 task done
[Installation]: vllm installation error installation Installation problems
#9305 opened Oct 12, 2024 by leoneyar
1 task done
1
[Bug]: Hermes 2 Pro Tool parser could not locate tool call start/end tokens in the tokenizer! bug Something isn't working
#9304 opened Oct 12, 2024 by LuckLittleBoy
1 task done
[New Model]: meta-llama/Llama-Guard-3-1B help wanted Extra attention is needed new model Requests to new models
#9294 opened Oct 11, 2024 by ayeganov
1 task done
[Bug]: Out of memory with large multi-step and large gpu-memory-utilization values - --num-scheduler-steps 16 --gpu-memory-utilization 0.941 bug Something isn't working
#9290 opened Oct 11, 2024 by varun-sundar-rabindranath
1 task done
4
[Bug]: Quantization example outdated (Ammo -> ModelOpt) bug Something isn't working
#9288 opened Oct 11, 2024 by kevalmorabia97
1 task done
May I ask what is the image parameter of vllm's api about blip2, and I have an error here，INFO: 127.0.0.1:40608 - "POST /v1/completions HTTP/1.1" 400 Bad Request usage How to use vllm
#9284 opened Oct 11, 2024 by zhaoxueqi6666
1
[Bug]: Simultaneous mm calls lead to permanently degraded performance. bug Something isn't working
#9283 opened Oct 11, 2024 by SeanIsYoung
1 task done
3
[Bug]: MiniCPM3-4B is support lora by --enable-lora ? bug Something isn't working
#9282 opened Oct 11, 2024 by ML-GCN
1 task done
1
5
[Bug]: VLLM doesn't support LoRa with config modules_to_save bug Something isn't working
#9280 opened Oct 11, 2024 by fahadh4ilyas
1 task done
4
[Usage]: Manually Increasing inference time usage How to use vllm
#9274 opened Oct 11, 2024 by Playerrrrr
1 task done
1
[Usage]: VLLM 0.6.2 includes vllm-flash-attn, is it no longer necessary to install flash-attn separately? usage How to use vllm
#9273 opened Oct 11, 2024 by Rssevenyu
[Usage]: blip2 inference code usage How to use vllm
#9270 opened Oct 11, 2024 by zhaoxueqi6666
1 task done
6
[Bug]: ptxas /tmp/tmpxft_002385ca_00000000-11_attention_kernels.compute_50.ptx, line 4986061; error : Feature 'f16 arithemetic and compare instructions' requires .target sm_53 or higher bug Something isn't working
#9269 opened Oct 11, 2024 by zhangfan-algo
1 task done
5
[RFC]: Make device agnostic for diverse hardware support RFC
#9268 opened Oct 11, 2024 by wangshuai09
1 of 25 tasks
@youkaichao
1
[Feature]: Improve Logging For Embedding Models feature request good first issue Good for newcomers
#9265 opened Oct 11, 2024 by robertgshaw2-neuralmagic
1 task done
1
[Bug]: AsyncLLMEngine stuck on a single too long request bug Something isn't working
#9263 opened Oct 10, 2024 by rickyyx
1 task done
1
[Bug]: Streaming response fails after one token (0.5.3.post1) bug Something isn't working
#9260 opened Oct 10, 2024 by NeonDaniel
1 task done
1
[Usage]: running gated models offline usage How to use vllm
#9255 opened Oct 10, 2024 by SamuelBG13
5
[Bug]: new beam search implementation ignores stop conditions bug Something isn't working
#9253 opened Oct 10, 2024 by nFunctor
1 task done
1
1
[Bug][Llama 3.2 Vision]: RuntimeWarning: coroutine 'AsyncMultiModalItemTracker.all_mm_data' was never awaited bug Something isn't working
#9249 opened Oct 10, 2024 by githebs
1 task done
[Installation]: build error from source on 4090 installation Installation problems
#9246 opened Oct 10, 2024 by ltm920716
1 task done
ProTip! Exclude everything labeled bug with -label:bug.