Commits

Commits on Jun 18, 2024

Create __init__.py (ModelTC#448 )
WANDY666
committedJun 18, 2024

Commits on Jun 17, 2024

remove return_logics params (ModelTC#446 )
hiworldwzj
committedJun 17, 2024
fix bug of mulit eos_id (ModelTC#442 )

shihaobai
and
baishihao
committedJun 17, 2024
fix mem manger get nccl_port way (ModelTC#440 )

hiworldwzj
and
wangzaijun
committedJun 17, 2024

Commits on Jun 14, 2024

fix log for mean_per_token_cost_time (ModelTC#438 )
hiworldwzj
committedJun 14, 2024

Commits on Jun 13, 2024

ppl w8a8 support splitfuse mode. (ModelTC#437 )

committedJun 13, 2024
add support for phi3-mini (ModelTC#433 )
shihaobai
committedJun 13, 2024
add high performance layernorm triton kernels. (ModelTC#432 )
flyinglandlord
committedJun 13, 2024

Commits on Jun 12, 2024

update tokenizer_mode help info. (ModelTC#431 )
hiworldwzj
committedJun 12, 2024
use triton to init window info (ModelTC#420 )
senbeiasano
committedJun 12, 2024
feat: re-impl the cohere with template (ModelTC#429 )
PannenetsF
committedJun 12, 2024

Commits on Jun 8, 2024

Update pre-commit.yml (ModelTC#428 )
hiworldwzj
committedJun 8, 2024
Update pre-commit.yml (ModelTC#427 )
hiworldwzj
committedJun 8, 2024
Update pre-commit.yml (ModelTC#426 )
hiworldwzj
committedJun 8, 2024
Update pre-commit.yml (ModelTC#425 )
hiworldwzj
committedJun 8, 2024
Create pre-commit.yml (ModelTC#424 )
hiworldwzj
committedJun 8, 2024
update llama wa quant (ModelTC#423 )

helloyongyang
and
hiworldwzj
committedJun 8, 2024

Commits on Jun 7, 2024

adapt qwen2 (ModelTC#421 )

shihaobai
and
baishihao
committedJun 7, 2024

Commits on Jun 4, 2024

Support Llava hf model (ModelTC#419 )

huochaitiantang
and
liuliang1
committedJun 4, 2024

Commits on Jun 3, 2024

add Visitor counter (ModelTC#418 )

shihaobai
and
baishihao
committedJun 3, 2024

Commits on Jun 2, 2024

Add triton w8a8 (ModelTC#417 )
helloyongyang
committedJun 2, 2024

Commits on May 27, 2024

Compat generate api (ModelTC#414 )

shihaobai
and
baishihao
committedMay 27, 2024

Commits on May 24, 2024

Dynamic img token (ModelTC#413 )

huochaitiantang
and
liuliang1
committedMay 24, 2024

Commits on May 20, 2024

Add tokens http interface (ModelTC#412 )

hiworldwzj
and
wangzaijun
committedMay 20, 2024
fix bug for w8a8 mode (ModelTC#411 )

hiworldwzj
and
wangzaijun
committedMay 20, 2024

Commits on May 15, 2024

【Feature】 Refactor the code and add support for multi-output features like beam search. (ModelTC#409 )

committedMay 15, 2024

Commits on Apr 15, 2024

bf16 compatibility for omitted kernel (ModelTC#396 )

committedApr 15, 2024

Commits on Apr 12, 2024

remove the support for triton==2.0.0 (ModelTC#395 )

hiworldwzj
and
wangzaijun
committedApr 12, 2024

Commits on Apr 11, 2024

Fix CI (ModelTC#392 )
llehtahw
committedApr 11, 2024
fix input_embdings.dtype=None.dtype (ModelTC#390 )
WANDY666
committedApr 11, 2024

Commits on Apr 10, 2024

Add bf16 inference for llm model (ModelTC#387 )

committedApr 10, 2024

Commits on Apr 9, 2024

Align generate and generate_stream api with tgi & fix internlm2 weights split size (ModelTC#385 )
huochaitiantang
committedApr 9, 2024

Commits on Apr 8, 2024

support llama-quik, a w4a4 quantization method (ModelTC#386 )

huakaigo
and
huakaigo
committedApr 8, 2024

Commits on Apr 1, 2024

add min_new_tokens sampling parameter (ModelTC#384 )
hiworldwzj
committedApr 1, 2024

Commits on Mar 29, 2024

w6a16 mode (ModelTC#382 )
hiworldwzj
committedMar 29, 2024