Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 15, 2024

• bf16 compatibility for omitted kernel (ModelTC#396)

  

  3 peopleauthored
  4b037f6

Commits on Apr 12, 2024

• remove the support for triton==2.0.0 (ModelTC#395)

  

  hiworldwzj

  and

  wangzaijun

  authored
  bcb3212

Commits on Apr 11, 2024

• Fix CI (ModelTC#392)

  llehtahw

  authored
  390ac96

• fix input_embdings.dtype=None.dtype (ModelTC#390)

  WANDY666

  authored
  aa2b655

Commits on Apr 10, 2024

• Add bf16 inference for llm model (ModelTC#387)

  

  3 peopleauthored
  15a050a

Commits on Apr 9, 2024

• Align generate and generate_stream api with tgi & fix internlm2 weights split size (ModelTC#385)

  huochaitiantang

  authored
  cf15435

Commits on Apr 8, 2024

• support llama-quik, a w4a4 quantization method (ModelTC#386)

  

  huakaigo

  and

  huakaigo

  authored
  8e34dad

Commits on Apr 1, 2024

• add min_new_tokens sampling parameter (ModelTC#384)

  hiworldwzj

  authored
  a231505

Commits on Mar 29, 2024

• w6a16 mode (ModelTC#382)

  hiworldwzj

  authored
  7cd10a0

Commits on Mar 26, 2024

• fix eos_id default value to [2] (ModelTC#378)

  hiworldwzj

  authored
  78047d1

• add sampling params: skip_special_tokens spaces_between_special_tokens (ModelTC#373)

  shihaobai

  authored
  5667c98

• Update model.py To Fix ntk length bug. (ModelTC#375)

  hiworldwzj

  authored
  aa98b35

Commits on Mar 25, 2024

• reset the ntk length range (ModelTC#374)

  

  shihaobai

  and

  baishihao

  authored
  f5dc783

Commits on Mar 21, 2024

• fix: intxweigtht -> wxa16, b_ready_cache_len -> infer_state.b_ready_c… (ModelTC#371)

  

  WANDY666

  and

  wanzihao

  authored
  755c4fd

• Bug Fix ModelTC#368 (ModelTC#370)

  

  hiworldwzj

  and

  wangzaijun

  authored
  c3dc640

• add cuda int4kv copy kernel (ModelTC#369)

  

  hiworldwzj

  and

  wangzaijun

  authored
  c5d6794

Commits on Mar 20, 2024

• fix: b_ready_cache_len default value (ModelTC#366)

  

  shihaobai

  and

  baishihao

  authored
  9fed5e9

• add mode ppl_int4kv_flashdecoding (ModelTC#367)

  hiworldwzj

  authored
  859a48b

Commits on Mar 19, 2024

• fix issue ModelTC#357 (ModelTC#365)

  flyinglandlord

  authored
  986f93d

• Update context_flashattention_nopad.py To fix triton 2.0.0 import error. (ModelTC#364)

  hiworldwzj

  authored
  56940dd

• add ppl int8kv flashdecoding mode (ModelTC#363)

  hiworldwzj

  authored
  c96e5d5

Commits on Mar 18, 2024

• To improve the code's compatibility with different PyTorch versions. (ModelTC#361)

  

  hiworldwzj

  and

  wangzaijun

  authored
  6d67fbb

• fix requirements.txt (ModelTC#360)

  hiworldwzj

  authored
  8debc65

• Fix bug for rope_scaling in config.josn is None (ModelTC#359)

  shihaobai

  authored
  537c871

• support length penalty (ModelTC#358)

  senbeiasano

  authored
  b445a6b

• [Feature] Dynamic prompt cache (ModelTC#356)

  

  6 peopleauthored
  f1f66fe

Commits on Mar 14, 2024

• Update silu_and_mul.py to fix the int32 overflow issue (ModelTC#354)

  hiworldwzj

  authored
  e2b5168

Commits on Mar 12, 2024

• bugfix: fix uncorrect quantize when using lmdeploy_w4a16 mode with bfloat16 model (ModelTC#352)

  

  3 peopleauthored
  5109f0b

Commits on Mar 7, 2024

• add support for starcoder2 (ModelTC#347)

  WuSiYu

  authored
  486f647

• [gemma_2b] support (ModelTC#346)

  senbeiasano

  authored
  adba526

• Add Qwen2 Model (ModelTC#340)

  

  flyinglandlord

  and

  hiworldwzj

  authored
  414aa08

Commits on Mar 6, 2024

• Support internlm2 wquant. (ModelTC#345)

  helloyongyang

  authored
  f7b9937

Commits on Mar 4, 2024

• Fix bug for rope_scaling in config.josn is None (ModelTC#344)

  hiworldwzj

  authored
  b1a73ec

• Fix weight only quant bug. (ModelTC#342)

  

  helloyongyang

  and

  hiworldwzj

  authored
  982e802

• add dynamic parse for internlm2 (ModelTC#341)

  

  shihaobai

  and

  baishihao

  authored
  01a87de

Pagination

PreviousNext