Commits

Commits on Apr 12, 2024

remove the support for triton==2.0.0 (ModelTC#395 )

hiworldwzj
and
wangzaijun
committedApr 12, 2024

Commits on Apr 1, 2024

add min_new_tokens sampling parameter (ModelTC#384 )
hiworldwzj
committedApr 1, 2024

Commits on Mar 29, 2024

w6a16 mode (ModelTC#382 )
hiworldwzj
committedMar 29, 2024

Commits on Mar 26, 2024

fix eos_id default value to [2] (ModelTC#378 )
hiworldwzj
committedMar 26, 2024
Update model.py To Fix ntk length bug. (ModelTC#375 )
hiworldwzj
committedMar 26, 2024

Commits on Mar 21, 2024

Bug Fix ModelTC#368 (ModelTC#370 )

hiworldwzj
and
wangzaijun
committedMar 21, 2024
add cuda int4kv copy kernel (ModelTC#369 )

hiworldwzj
and
wangzaijun
committedMar 21, 2024

Commits on Mar 20, 2024

add mode ppl_int4kv_flashdecoding (ModelTC#367 )
hiworldwzj
committedMar 20, 2024

Commits on Mar 19, 2024

Update context_flashattention_nopad.py To fix triton 2.0.0 import error. (ModelTC#364 )
hiworldwzj
committedMar 19, 2024
add ppl int8kv flashdecoding mode (ModelTC#363 )
hiworldwzj
committedMar 19, 2024

Commits on Mar 18, 2024

To improve the code's compatibility with different PyTorch versions. (ModelTC#361 )

hiworldwzj
and
wangzaijun
committedMar 18, 2024
fix requirements.txt (ModelTC#360 )
hiworldwzj
committedMar 18, 2024
[Feature] Dynamic prompt cache (ModelTC#356 )

committedMar 18, 2024

Commits on Mar 14, 2024

Update silu_and_mul.py to fix the int32 overflow issue (ModelTC#354 )
hiworldwzj
committedMar 14, 2024

Commits on Mar 4, 2024

Fix bug for rope_scaling in config.josn is None (ModelTC#344 )
hiworldwzj
committedMar 4, 2024

Commits on Feb 19, 2024

Update test_settings.py (ModelTC#331 )
hiworldwzj
committedFeb 19, 2024

Commits on Feb 5, 2024

fix support for model_type internlm2 (ModelTC#328 )
hiworldwzj
committedFeb 5, 2024

Commits on Jan 25, 2024

[Update] KV Merge And FFn up gate merge. (ModelTC#320 )

committedJan 25, 2024

Commits on Jan 16, 2024

fix bug when vob_size % world_size != 0 (ModelTC#300 )
hiworldwzj
committedJan 16, 2024

Commits on Jan 10, 2024

add ppl_fp16_flashdecoding kernel support (ModelTC#292 )
hiworldwzj
committedJan 10, 2024

Commits on Jan 9, 2024

Revert "add ceph-based model loading" (ModelTC#289 )
hiworldwzj
committedJan 9, 2024

Commits on Jan 5, 2024

fix for torch 1.13 (ModelTC#283 )
hiworldwzj
committedJan 5, 2024

Commits on Jan 4, 2024

add mulitmodal_emb kernel (ModelTC#278 )

hiworldwzj
and
wangzaijun
committedJan 4, 2024

Commits on Dec 28, 2023

Fix (ModelTC#270 )

hiworldwzj
and
wangzaijun
committedDec 28, 2023

Commits on Dec 25, 2023

[feature] long_truncation_mode supported (ModelTC#265 )

hiworldwzj
and
wangzaijun
committedDec 25, 2023

Commits on Dec 22, 2023

fix (ModelTC#261 )
hiworldwzj
committedDec 22, 2023

Commits on Dec 20, 2023

fix log (ModelTC#257 )
hiworldwzj
committedDec 20, 2023
fix log. (ModelTC#256 )
hiworldwzj
committedDec 20, 2023

Commits on Dec 19, 2023

Update to make splitfuse_mode can run rightly when using triton-nightly. (ModelTC#254 )
hiworldwzj
committedDec 19, 2023
[Feature] support to output all prompt logprobs. (ModelTC#253 )

hiworldwzj
and
wangzaijun
committedDec 19, 2023

Commits on Dec 14, 2023

Upgrade gqa attention kernel. (ModelTC#247 )
hiworldwzj
committedDec 14, 2023

Commits on Dec 4, 2023

upgrade gqa kernel to mode setting. (ModelTC#242 )

hiworldwzj
and
wangzaijun
committedDec 4, 2023

Commits on Dec 1, 2023

support ppl fp16 attention (ModelTC#235 )
hiworldwzj
committedDec 1, 2023

Commits on Nov 30, 2023

upgrade router len calcu way. (ModelTC#232 )

hiworldwzj
and
wangzaijun
committedNov 30, 2023

Commits on Nov 29, 2023

[feature] GQA triton flashdecoding kernel (ModelTC#231 )

hiworldwzj
and
wangzaijun
committedNov 29, 2023