Skip to content

Issues: InftyAI/llmaz

Milestone v0.1.0
#63 opened Aug 5, 2024 by kerthcet
Open
[WebUI] Add support for webui
#160 opened Sep 13, 2024 by kerthcet
Open 2
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

38 Open 41 Closed
38 Open 41 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add TensorRT-LLM support as another backend needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#205 opened Nov 18, 2024 by kerthcet
3 tasks
Install lws controller together with llmaz controller in the same namespace cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#202 opened Nov 13, 2024 by kerthcet
3
Support speculative decoding with llama.cpp needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#197 opened Nov 5, 2024 by kerthcet
3 tasks
Serverless support needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#192 opened Oct 29, 2024 by kerthcet
3 tasks done
Support to serving Stable Diffusion models needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#189 opened Oct 23, 2024 by kerthcet
1 of 3 tasks
Is there any early proposal or document about integrating with Gateway API ? feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#165 opened Sep 15, 2024 by caozhuozi
2
[WebUI] Add support for webui feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#160 opened Sep 13, 2024 by kerthcet
3 tasks
2
[Umbrella] Improve test coverages cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. good first issue Good for newcomers needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#156 opened Sep 12, 2024 by kerthcet
4
Customized flags for backendRuntimes feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#140 opened Sep 11, 2024 by kerthcet
3 tasks done
2
Support traditional models feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#133 opened Sep 9, 2024 by kerthcet
3 tasks done
1
Loading model weights more efficiently feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#119 opened Sep 2, 2024 by kerthcet
3 tasks
v0.1.0
@kerthcet
6
Support scaling with Spot instances for cost saving needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one. question Further information is requested
#106 opened Aug 27, 2024 by kerthcet
3 tasks
6
Support filesystems feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#100 opened Aug 19, 2024 by kerthcet
3 tasks done
1
Model aware scheduling feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#96 opened Aug 19, 2024 by kerthcet
2 of 3 tasks
3
Prompts managements feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#90 opened Aug 17, 2024 by kerthcet
3 tasks done
1
Failover policy for various backends important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. needs-kind Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#86 opened Aug 13, 2024 by kerthcet
2 of 3 tasks
1
Parallel model serving important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. needs-kind Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#85 opened Aug 13, 2024 by kerthcet
2 of 3 tasks
1
Lack the flexibility to express deploy primitives needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one. question Further information is requested
#81 opened Aug 12, 2024 by kerthcet
4
Integrate with Kueue for fungibility capacity feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#74 opened Aug 8, 2024 by kerthcet
2 of 3 tasks
1
Mount /dev/shm for shared memory files feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#73 opened Aug 8, 2024 by kerthcet
3 tasks
1
Benchmark toolkit support feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#66 opened Aug 6, 2024 by kerthcet
1 of 3 tasks
3
Milestone v0.1.0 needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#63 opened Aug 5, 2024 by kerthcet
Support different GPU accelerators for fungibility feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#62 opened Aug 5, 2024 by kerthcet
1 of 3 tasks
v0.1.0
1
An an example for multi-host inference with Service cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#61 opened Aug 5, 2024 by kerthcet
3 tasks
v0.1.0
1
Model version management feature Categorizes issue or PR as related to a new feature. important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. needs-triage Indicates an issue or PR lacks a label and requires one.
#58 opened Aug 4, 2024 by kerthcet
3 tasks done
2
ProTip! Updated in the last three days: updated:>2024-11-24.