This repository presents implementation of our work. We introduce Concept Depth🔍 to measure where a concept is learned by LLMs with different sizes.

Paper Link: https://arxiv.org/abs/2404.07066

This novel concept is used for analyze the comprehension ability of Large Language Model (LLMs), as well as the difficulty of a concept's understanding. We use probing techniques🔍 on each layer's parameters to detect the layer accuracy, F1-score, and AUC of classification task.

• LLMs tend to effectively categorize simple (low-level) tasks, indicating that these concepts are learned in the first few initial layers.

• Difficult (High-level) tasks may only be recognizable (if at all) in deeper layers, and LLMs of the same size perform largely consistently across datasets in terms of concept depth.

pip install -r requirements.txt

We apply 9 datasets to evaluate Concept Depth.

Raw Files

See: ./dataset

Dataset Size

Here shows the amount of data samples we use to train the LLMs in each dataset.

Note: if your huggingface token is not ready, please try the code below:

from huggingface_hub import notebook_login
notebook_login()

Run by Jupyter Please open run.ipynb for jupyter environment.

Run by One-click

python run.py

Run by Instructions

python3 main.py --model $MODEL --quant QUANT --noise NOISE --dataset $DATASET

$MODEL: Model name.

$DATASET: Dataset name.

QUANT: The number of bits for quantization. Could be 8, 16, and 32.

NOISE: Replace NOISE with "non-noise" or "noise". "non-noise" for no noise perturbation. "noise" for noise perturbation. Please remove the quotation marks when replacing.

Feel free to change these 4 variables. For example: If we want to train Gemma-2B over dataset STSA, the instruction is:

python3 main.py --model google/gemma-2b --dataset STSA

If we want to train Gemma-2B over dataset STSA, using 16 bit quantization, the instruction is:

python3 main.py --model google/gemma-2b --dataset STSA --quant 16 

If we want to train Gemma-2B over dataset STSA with noise perturbation, the instruction is:

python3 main.py --model google/gemma-2b --dataset STSA --noise noise

Other cases could be found here.

Please cite the following paper corresponding to the repository:

@misc{jin2024exploring,
      title={Exploring Concept Depth: How Large Language Models Acquire Knowledge at Different Layers?}, 
      author={Mingyu Jin and Qinkai Yu and Jingyuan Huang and Qingcheng Zeng and Zhenting Wang and Wenyue Hua and Haiyan Zhao and Kai Mei and Yanda Meng and Kaize Ding and Fan Yang and Mengnan Du and Yongfeng Zhang},
      year={2024},
      eprint={2404.07066},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}