Commits

Commits on Oct 27, 2023

[Feature] Add int8 and int4 quantize for starcoder. (ModelTC#179 )

WANDY666
and
wanzihao
committedOct 27, 2023

Commits on Oct 26, 2023

fix bug for qwen tp 2 setting (ModelTC#181 )
hiworldwzj
committedOct 26, 2023

Commits on Oct 23, 2023

fix bug for /v1/chat/completions (ModelTC#175 )
hiworldwzj
committedOct 23, 2023

Commits on Oct 17, 2023

fix the invalid fig (ModelTC#167 )
shihaobai
committedOct 17, 2023

Commits on Oct 16, 2023

GitHub main (ModelTC#166 )

committedOct 16, 2023
Revert "Merge internal (ModelTC#164 )" (ModelTC#165 )
shihaobai
committedOct 16, 2023
Merge internal (ModelTC#164 )

shihaobai
and
baishihao
committedOct 16, 2023

Commits on Oct 12, 2023

update the docker start command (ModelTC#159 )
shihaobai
committedOct 12, 2023

Commits on Sep 25, 2023

Update README.md (ModelTC#151 )
hiworldwzj
committedSep 25, 2023
Update Readme.md (ModelTC#150 )
hiworldwzj
committedSep 25, 2023
modify setup.py triton version (ModelTC#149 )
hiworldwzj
committedSep 25, 2023

Commits on Sep 22, 2023

Fix internlm bias (ModelTC#143 )

ChieloNewctle
and
hiworldwzj
committedSep 22, 2023
[Quant] INT4 weight using different kernel when prefill & decode. (ModelTC#144 )

Tracin
and
zhangqi3
committedSep 22, 2023

Commits on Sep 21, 2023

starcoder support ppl int8kv (ModelTC#141 )

hiworldwzj
and
wangzaijun
committedSep 21, 2023

Commits on Sep 20, 2023

fix bug for qwen7b 8k length (ModelTC#140 )
hiworldwzj
committedSep 20, 2023
qwen7b to support 8k length (ModelTC#139 )

hiworldwzj
and
wangzaijun
committedSep 20, 2023

Commits on Sep 19, 2023

add stop_sequences way to stop generate tokens (ModelTC#137 )

hiworldwzj
and
wangzaijun
committedSep 19, 2023

Commits on Sep 18, 2023

add llama2_ppl model, that use ppl int8kv kernel (ModelTC#136 )
hiworldwzj
committedSep 18, 2023
Some API changes (ModelTC#125 )

yunfeng-scale
and
hiworldwzj
committedSep 18, 2023
Support Llama Multimodal (ModelTC#133 )

huochaitiantang
and
liuliang1
committedSep 18, 2023

Commits on Sep 14, 2023

add a llama_ppl model that use ppl fast decode attention kernel. (ModelTC#132 )
hiworldwzj
committedSep 14, 2023

Commits on Sep 12, 2023

[Feature] Add int8 quantize for llama. (ModelTC#112 )

Tracin
and
zhangqi3
committedSep 12, 2023
add support dynamic ntk for llama (ModelTC#127 )
hiworldwzj
committedSep 12, 2023

Commits on Sep 11, 2023

Support OpenAI-Compatible APIs (ModelTC#123 )
WANDY666
committedSep 11, 2023

Commits on Sep 8, 2023

[Support to CodeLlama] Add rope_theta to llama models (ModelTC#118 )
ChieloNewctle
committedSep 8, 2023

Commits on Sep 5, 2023

Uniform code style for utils for better readability (ModelTC#108 )
UranusSeven
committedSep 5, 2023

Commits on Aug 31, 2023

[Bug Fix] Fix bug for triton 2.1.0 llama decode kernel and add llama2 decode attention kernel. (ModelTC#113 )
hiworldwzj
committedAug 31, 2023

Commits on Aug 28, 2023

update llama attention decode kernel support for triton2.1.0, better performance (ModelTC#105 )
hiworldwzj
committedAug 28, 2023

Commits on Aug 24, 2023

fix the rotary for the chatglm2 (ModelTC#103 )
shihaobai
committedAug 24, 2023
Add id and logprob to generate_stream (ModelTC#99 )
ChieloNewctle
committedAug 24, 2023

Commits on Aug 23, 2023

Add a cn doc about how to add a new model (ModelTC#98 )
hiworldwzj
committedAug 23, 2023

Commits on Aug 22, 2023

Internlm (ModelTC#96 )

shihaobai
and
baishihao
committedAug 22, 2023
Support for NTK-aware Scaled RoPE from Environment Variables (ModelTC#93 )
singularity-s0
committedAug 22, 2023

Commits on Aug 21, 2023

Fix installer not copying all files (ModelTC#94 )
singularity-s0
committedAug 21, 2023
fix bug for aborted req (ModelTC#91 )
hiworldwzj
committedAug 21, 2023