World model games are awesome!
Here we collect papers, open-source projects, and applications of such interactive video models.
Click the badges below the heading to visit the project's relevant links.
The badge indicates an estimate of the frames per second of the model when run on an NVIDIA RTX 4090 GPU at reasonable inference settings. Anything below 1 FPS will be highlighted in red
. In case of
, model performance on an RTX 4090 has not yet been reported.
Contributions are very welcome!
Open-source training and inference, initially launched with 26 atari games and CS:GO.
100M models of 26 Atari games.
A 500M model and inference code are open-source.
One size of the model and inference code are open-source.
2.7B model trained on data from Forza Horizon 5 and Cyberpunk 2077 as well as real-world footage.
It runs Doom! The grandfather paper of the field.
Seminal paper generalising world models to different games (and pictures!), focused on 2D platformers.
An early model trained on a 4th Generation Pokemon game.
Game Papers:
- The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control - Matrix Paper (Nov 2024).
- Diffusion for World Modeling: Visual Details Matter in Atari - Diamond Paper (Oct 2024).
- Diffusion Models are Real-time Game Engines - GameNGen Paper (Aug 2024).
- Genie: Generative Interactive Environments - Genie Paper (Feb 2024).