BLAST: Block Level Adaptive Structured Matrix for Efficient Deep Neural Network Inference

Changwoo Lee, Soo Min Kwon, Qing Qu, and Hun-Seok Kim

This repo is being actively updated.

• The paper is accepted to NeurIPS 2024.

The packages can be installed via conda env create --file environment.yml.

Additionally, install lm-evaluation-harness with BLAST implementation via

cd lm-evaluation-harness
pip install -e .

Run bash ./scripts/decompose_llama.sh 0-31.

Run bash ./scripts/train_blast.sh. The script assumes that 4 gpus are available.

Run bash scripts/lm-eval-blast.sh.

This repo is highly inspired by huggingface/transformers.

Please cite our paper if you find this repo or our paper useful

@inproceedings{
    lee2024blast,
    title={{BLAST}: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference},
    author={Lee, Changwoo and Kwon, Soo Min and Qu, Qing and Kim, Hun-Seok},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
}