[NeurIPS 2023] Learning Transformer Programs

Code behind the work "Single Cortical Neurons as Deep Artificial Neural Networks", published in Neuron 2021

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

Convert Machine Learning Code Between Frameworks