1 Pull request merged by 1 person

• [Fix] Fix the Error of q, k, and v states must have the same dtype when using flash attention forward.

  #15 merged May 30, 2024

1 Issue closed by 1 person

• Possibility to integrate multiple model types?

  #16 closed May 29, 2024

2 Issues opened by 1 person

• Add shortcut for MLP hidden state computation.

  #18 opened Jun 14, 2024

• [Feature] Support New Arguments for Expert Routing Policies.

  #17 opened May 31, 2024

1 Unresolved conversation

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• LoRA MoE with k_proj, up_proj, down_proj

  #13 commented on Jun 6, 2024 • 2 new comments