Vulkan Mixture of Experts (MoE) support (llama/7628) · ggerganov/ggml@1981d5b · GitHub

Commit

Vulkan Mixture of Experts (MoE) support (llama/7628)

* Finish Vulkan mul_mat_id implementation

* Add Vulkan sum_rows and div ops

* Fix MUL_MAT_ID matrix matrix shader

* Fix MUL_MAT_ID matrix vector shader dispatch size

* Fix MUL_MAT_ID matrix vector shader and dispatch code

* Update Vulkan CPU offload for MUL_MAT_ID

* Fix crash when using split mode none and setting a main GPU

Loading branch information

0cc4m authored and ggerganov committed Jun 15, 2024

1 parent 5ed1871 commit 1981d5b