This repository is the official implementation of the paper “Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective”, based on the implementation of minGPT in PyTorch.

@misc{feng2023revealing,
      title={Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective}, 
      author={Guhao Feng and Bohang Zhang and Yuntian Gu and Haotian Ye and Di He and Liwei Wang},
      year={2023},
      eprint={2305.15408},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

The training and testing data can be generated via the command in DATA.md.

The training instruction is in TRAIN.md.

The testing instruction is in TEST.md.

To faciliate future research, we upload all training logs in the logs folder.

This project is under the MIT license. See LICENSE for details.