llama+gptq has multiple open source impelementations, it is like hello world for NLP, just like lenet MINST for image classification. This makes learning how to re-implement llama+gptq on other programming api much easier. The goal is to learn the GPU programming stack especially the profilers, using llama+gptq as an learning exercise.
- https://jaykmody.com/blog/gpt-from-scratch/
- https://github.com/lucidrains/x-transformers
- https://github.com/karpathy/nanoGPT
- https://github.com/facebookresearch/llama
- https://github.com/ggerganov/llama.cpp
- https://github.com/turboderp/exllama
- https://github.com/Lightning-AI/lit-llama
- https://github.com/huggingface/transformers/tree/main/src/transformers/models/llama
- https://github.com/marella/ctransformers/blob/main/models/llms/llama.cc
- https://github.com/OpenNMT/CTranslate2/blob/master/python/ctranslate2/converters/transformers.py
- https://github.com/kayvr/token-hawk
- https://github.com/NolanoOrg/sparse_quant_llms/blob/main/llama_model.py
- https://github.com/rustformers/llm/blob/main/crates/models/llama/src/lib.rs
- https://github.com/thisserand/FastChat/blob/4a57c928a906705404eae06f7a44b4da45828487/fastchat/train/llama_flash_attn_monkey_patch.py
- https://github.com/Sea-Snell/JAX_llama
- https://github.com/mlc-ai/mlc-llm/blob/main/mlc_llm/relax_model/llama.py
- https://github.com/juncongmoo/pyllama/blob/main/llama/hf/modeling_llama.py
- https://github.com/recmo/cria/blob/main/cria.py
- https://github.com/ypeleg/llama/blob/master/llama/modeling_llama.py
- https://github.com/gotzmann/llama.go/blob/main/pkg/llama/llama.go
- https://github.com/zphang/minimal-llama/blob/main/minimal_llama/model.py
- https://github.com/jankais3r/LLaMA_MPS/blob/main/llama/model.py
- https://github.com/young-geng/EasyLM/blob/main/EasyLM/models/llama/llama_model.py
- https://github.com/davisyoshida/llama-haiku/blob/master/llama_haiku/model.py
- https://github.com/p-nordmann/eqx-llama/blob/master/eqx_llama/model.py
- https://github.com/Noeda/rllama
- https://github.com/tuxifan/llama.cpp-kompute/tree/kompute
- https://github.com/tinygrad/tinygrad/blob/master/examples/llama.py
- https://github.com/tpoisonooo/llama.onnx
- https://github.com/jmorganca/ollama/blob/main/llama/llama.go
- https://github.com/karpathy/llama2.c
- https://github.com/ayaka14732/llama-2-jax
- https://github.com/srush/llama2.rs
- https://github.com/jrudolph/llama2.scala
- https://blog.briankitano.com/llama-from-scratch/
- https://github.com/srush/llama2.rs/
- https://github.com/tairov/llama2.mojo/blob/master/llama2.mojo
- https://github.com/IST-DASLab/gptq
- https://github.com/turboderp/exllama
- https://github.com/qwopqwop200/GPTQ-for-LLaMa
- https://github.com/PanQiWei/AutoGPTQ
- https://github.com/fpgaminer/GPTQ-triton
- https://github.com/Lightning-AI/lit-llama/blob/main/lit_llama/quantization.py
- https://github.com/mlc-ai/mlc-llm/blob/main/mlc_llm/quantization/autogptq_quantization.py
- https://github.com/LeiWang1999/AutoGPTQ.tvm
- https://github.com/K024/chatglm-q/blob/main/chatglm_q/int4/triton_ops.py
- https://github.com/3outeille/GPTQ-for-RWKV/blob/master/quant_cuda_kernel.cu
- https://github.com/davisyoshida/easy-lora-and-gptq
- https://github.com/davisyoshida/jax-gptq
- https://github.com/cannstandard/gptq-modal/blob/main/gptq_wrapper.py
- https://github.com/thisserand/FastChat/blob/main/fastchat/serve/load_gptq_model.py
- https://github.com/juncongmoo/pyllama/blob/main/llama/llama_quant.py
====
拉一个民用设备跑大模型的微信群。入群条件提供你在 https://www.reddit.com/r/LocalLLaMA/ 上发的帖子或者评论的截图，然后发给我的微信号（bmN0YW93ZW4=）。