This repository provides a list of papers that I believe are interesting and influential on the Free-Energy-Principle, or in Active Inference. If you believe I have missed any papers, please contact me at [email protected] or make a pull request with the information about the paper. I will be happy to include it.
This list is of papers focused specifically on the abstract mathematical formulation of the Free-Energy-Principle (FEP). The FEP is a theory which tries to determine the behaviours a non-equilibrium thermodynamical system must exhibit if it is to maintain itself as a separate entity over time. It argues that any such system must minimize a quantity called the free energy and that, over the course of this minimisation, behaviour much like action and perception must emerge.
The key prerequisites for the FEP is that a 'system' has a special kind of statistical separation from the world called a Markov Blanket, which it must maintain if it is to remain a system, and that the system possesses a non-equilibrium steady state which it self-organises to and tries to maintain over time against the dissipative forces of entropy.
Much of the work in the FEP has been applying its general tenets to understand biological far-from-equilibrium systems, especially the brain.
If you are just starting out, I reccomend reading all the papers in the 'Survey' section in order. These are all great tutorials or overviews which should give you a great grounding in the intuitions of the theory, and then the later two tutorials should start building up much of the mathematical core of the theory (especially around predictive coding).
- What does the free energy principle tell us about the brain? , (2019) by Samuel J Gershman [bib]
This provides a great high level introduction to the basic ideas and intuitions of the FEP, with a small amount of crucial mathematical background.
- The free-energy principle: a unified brain theory? , (2010) by Karl Friston [bib]
This provides a great overview for the initial intuitions behind the FEP and its application to the brain.
- A tutorial on the free-energy framework for modelling perception and learning , (2017) by Rafal Bogacz [bib]
This is a great review which introduces the basics of predictive coding and the FEP, including the maths and contains MATLAB sample code. If you want to start seriously diving into the maths, I would start here.
- The free energy principle for action and perception: A mathematical review , (2017) by Christopher L Buckley and Chang Sub Kim and Simon McGregor and Anil K Seth [bib]
This is a fantastic review which presents a complete walkthrough of the mathematical basis of the Free Energy Principle and Variational Inference, and derives predictive coding and (continuous time and state) active inference. I would reccomend reading this after Bogacz' tutorial (although be prepared -- it is a long and serious read)
- A Step-by-Step Tutorial on Active Inference and its Application to Empirical Data , (2021) by Ryan Smith and Karl Friston and Christopher Whyte [bib]
A detailed and clear walkthrough of discrete-state-space active inference, including detailed MATLAB code for a sample implementation.
- A free energy principle for a particular physics , (2019) by Karl Friston [bib]
This is Karl's magisterial monograph, and contains the most comprehensive description of the FEP to date
- A free energy principle for the brain , (2006) by Karl Friston and James Kilner and Lee Harrison [bib]
Perhaps the earliest paper describing the FEP. Provides a great description of the fundamental intuitions behind the theory (in needs of living systems to reduce their internal entropy to keep conditions within homeostatic bounds)
- A theory of cortical responses , (2005) by Karl Friston [bib]
An early but complete description of predictive coding as an application of the FEP and variational inference under Gaussian and Laplace assumptions. Also surprisingly readable. This is core reading on predictive coding and the FEP
- Learning and inference in the brain , (2003) by Karl Friston [bib]
- Reinforcement learning or active inference? , (2009) by Karl J Friston and Jean Daunizeau and Stefan J Kiebel [bib]
The earliest paper (I think) on active inference. Introduces the motivation behind the continuous state and time formulation of active inference. Shows how predictive coding can be used to learn actions as well as observations (by treating them the same)
- Action understanding and active inference , (2011) by Karl Friston and J{'e}r{'e}mie Mattout and James Kilner [bib]
Goes deep into the neuroscientific intuitions behind why you might want to think about action as a predicted observation and not a latent variable for biological brains. Presents Karl's view that action happens primarily at the periphery through simple 'reflex arcs' while all the real work is done by the generative models generating predictions.
- A free energy principle for biological systems , (2012) by Friston Karl [bib]
- Of woodlice and men , (2018) by Martin Fortier and Daniel A Friedman [bib]
A great interview with Karl. Goes into a lot of his personal motivations underlying his work on the FEP. I would recommend this perhaps as an initial place to start out if you know nothing of the FEP to grasp the underlying motivations of what it is trying to explain.
- The history of the future of the Bayesian brain , (2012) by Karl Friston [bib]
- Free energy, value, and attractors , (2012) by Karl Friston and Ping Ao [bib]
Mathematical paper by Karl and Ping Ao which begins fleshing out formally the notion of desires as attractors
- Attention, uncertainty, and free-energy , (2010) by Harriet Feldman and Karl Friston [bib]
Makes a conjectured link between precision in predictive coding and attention in the brain.
- Hierarchical models in the brain , (2008) by Karl Friston [bib]
Presents the 'full-construct' predictive coding model with both hierarchies and generalised coordinates.
- DEM: a variational treatment of dynamic systems , (2008) by Karl J Friston and N Trujillo-Barreto and Jean Daunizeau [bib]
Extends predictive coding to generalised coordinates, and derives the necessary inference algorithms for working with them -- i.e. DEM, dynamic expectation maximisation.
- Generalised filtering , (2010) by Karl Friston and Klaas Stephan and Baojuan Li and Jean Daunizeau [bib]
- Surfing uncertainty: Prediction, action, and the embodied mind , (2015) by Andy Clark [bib]
- Variational filtering , (2008) by Karl J Friston [bib]
Foundational treatment of variational inference for dynamical systems, as represented in generalised coordinates. Also relates variational filtering to other non-variational schemes like particle filtering and Kalman filtering.
- Action and behavior: a free-energy formulation , (2010) by Karl J Friston and Jean Daunizeau and James Kilner and Stefan J Kiebel [bib]
- The Markov Blanket Trick: On the Scope of the Free Energy Principle and Active Inference , (2021) by Vicente Raja and Dinesh Valluri and Edward Baggs and Anthony Chemero and Michael L Aderson [bib]
- How particular is the physics of the Free Energy Principle? , (2021) by Miguel Aguilera and Beren Millidge and Alexander Tschantz and Christopher L Buckley [bib]
This paper critically analyses and deconstructs various philosophical claims about what the FEP is saying. Specifically, it argues that there is not necessarily a connection between the statistical notion of a Markov Blanket, and a functional notion, meaning that an actual dynamical separation (such as a cell membrane) does not necessarily imply a statistical separation in the form of a Markov Blanket and vice versa. Secondly, it demonstrates and clarifies that the FEP only makes claims about the flow of internal states on average over counterfactual realizations of the system, and therefore the FEP cannot describe the individual trajectories of a system in terms of free energy minimization.
- A tale of two densities: Active inference is enactive inference , (2020) by Maxwell JD Ramstead and Michael D Kirchhoff and Karl J Friston [bib]
- Answering Schr{"o}dinger's question: A free-energy formulation , (2018) by Maxwell James D{'e}sormeau Ramstead and Paul Benjamin Badcock and Karl John Friston [bib]
- Thinking through other minds: A variational approach to cognition and culture , (2020) by Samuel PL Veissi{`e}re and Axel Constant and Maxwell JD Ramstead and Karl J Friston and Laurence J Kirmayer [bib]
- TTOM in action: Refining the variational approach to cognition and culture , (2020) by Samuel PL Veissi{`e}re and Axel Constant and Maxwell JD Ramstead and Karl J Friston and Laurence J Kirmayer [bib]
- What does the free energy principle tell us about the brain? , (2019) by Samuel J Gershman [bib]
This provides a great high level introduction to the basic ideas and intuitions of the FEP, with a small amount of crucial mathematical background.
- The anticipating brain is not a scientist: the free-energy principle from an ecological-enactive perspective , (2018) by Jelle Bruineberg and Julian Kiverstein and Erik Rietveld [bib]
- Predictive processing and the representation wars , (2018) by Daniel Williams [bib]
- Whatever next? Predictive brains, situated agents, and the future of cognitive science , (2013) by Andy Clark [bib]
- Predictions in the eye of the beholder: an active inference account of Watt governors , (2020) by Manuel Baltieri and Christopher L Buckley and Jelle Bruineberg [bib]
- From allostatic agents to counterfactual cognisers: active inference, biological regulation, and the origins of cognition , (2020) by Andrew W Corcoran and Giovanni Pezzulo and Jakob Hohwy [bib]
- Interoceptive inference, emotion, and the embodied self , (2013) by Anil K Seth [bib]
- Active interoceptive inference and the emotional brain , (2016) by Anil K Seth and Karl J Friston [bib]
- The cybernetic Bayesian brain , (2014) by Anil K Seth [bib]
- Presence, objecthood, and the phenomenology of predictive perception , (2015) by Anil K Seth [bib]
- The Math is not the Territory: Navigating the Free Energy Principle , (2020) by Mel Andrews [bib]
- Life as we know it , (2013) by Karl Friston [bib]
A heuristic demonstration of the concept that Karl will later refer to as 'Bayesian mechanics', this paper surveys the notion that any random dynamical systems with the right kind of coupling among its sub-systems (i.e. a Markov blanket), will naturally appear as if it's performing a kind of approximate Bayesian inference. This argument is motivated by appeal to the existence of a non-equilibrium steady-state density, to which the system's probability distribution converges over time.
- Knowing one's place: a free-energy approach to pattern regulation , (2015) by Karl Friston and Michael Levin and Biswa Sengupta and Giovanni Pezzulo [bib]
- Morphogenesis as Bayesian inference: A variational approach to pattern formation and control in complex biological systems , (2019) by Franz Kuchling and Karl Friston and Georgi Georgiev and Michael Levin [bib]
- Neural and phenotypic representation under the free-energy principle , (2020) by Maxwell JD Ramstead and Casper Hesp and Alexander Tschantz and Ryan Smith and Axel Constant and Karl Friston [bib]
- Parcels and particles: Markov blankets in the brain , (2020) by Karl J Friston and Erik D Fagerholm and Tahereh S Zarghami and Thomas Parr and In{^e}s Hip{'o}lito and Lo{"\i}c Magrou and Adeel Razi [bib]
- Markov blankets in the brain , (2020) by Ines Hipolito and Maxwell Ramstead and Laura Convertino and Anjali Bhat and Karl Friston and Thomas Parr [bib]
- Modules or Mean-Fields? , (2020) by Thomas Parr and Noor Sajid and Karl J Friston [bib]
The 'free energy' response to the Fodorian notion of 'modularity' as an explanation of functional segregation, here motivated by an appeal to the stochastic dynamics of Markov-blanketed systems. Parr et al. argue that, given a particular conditional independency structure among the components that comprise a random dynamical system, one can interpret the system and its dynamics as entertaining a mean-field factorised generative model of its local environment, as opposed to appealing to philosophically or otherwise-unsatisfying notions such as 'modularity'.
- Biological self-organisation and Markov blankets , (2017) by Ensor Rafael Palacios and Adeel Razi and Thomas Parr and Michael Kirchhoff and Karl Friston [bib]
- The Emperor’s New Markov Blankets , (2020) by Jelle Bruineberg and Krzysztof Dolega and Joe Dewhurst and Manuel Baltieri [bib]
- Markov blankets, information geometry and stochastic thermodynamics , (2020) by Thomas Parr and Lancelot Da Costa and Karl Friston [bib]
This paper gives succinct and schematic treatments of several of the main concepts explored in a Free Energy Principle for a Particular Physics, particularly those related to Bayesian mechanics and information geometry. This work importantly delineates some of the conditions required of a system, so that its internal states approximately parameterise beliefs about external states. Fluctuation theorems are also invoked to relate the probability of trajectories or sequences of states to existing concepts in the active inference world, such as information gain, risk, and ambiguity resolution.
Active Inference is a process theory of neurobiological function inspired by and closely related to the FEP. However Active Inference stands independent of the FEP and can be true even if the FEP is not, and similarly can potentially be falsified without impacting the FEP. The core idea behind Active Inference is the idea that the brain performs both action and perception by variational inference on a unified objective function.
In effect, the key idea behind active inference is that our brains possess powerful probabilistic generative models and inference engines, and that to select actions, we repurpose this powerful capacity we use for perception to also infer potential actions. Hence Active Inference.
This high-level description leaves open the exact type of models and inference being used for action inference in the brain. The active inference literature contains three clear strands of work, which correspond to different assumptions on the exact form of generative models which are proposed to be utilized by the brain. Discrete active inference focuses on models of discrete state-spaces parametrised by categorical distributions and transition matrices. Continuous active inference focuses on the continuous time case with (generally) linear dynamics, and Deep active inference focuses on using deep neural networks to 'scale up' active inference by amortising probabilistic distributions with learned maps. The discrete-state-space work has close similarities with bandit-problems and neuroscience tasks and forms a tractable test-bed to understand different kinds of behaviour. Most of the work of creating active inference models of brain function (or dysfunction) lies within this paradigm. Continuous active inference, which is being used for robot control, has close links to classical control theory, while Deep active inference has close links with reinforcement learning and machine learning.
The task of inferring actions (requiring detailed models of future outcomes given these actions), is a subtly more complex task than simply inferring the immediate causes of sensory data as in perceptual inference. It therefore requires different objective functionals (the expected free energy) and potentially more advanced message-passing inference algorithms. This work is summarised in the 'Message Passing and Free Energies' section.
- Surveys and Tutorials
- Discrete State Space Formulation
- Continuous Time Formulation
- Message Passing and Free Energies
- Active Inference for Control Theory/Robotics
- Neuroscience and Computational Psychiatry Applications
- Deep Active Inference
- Active inference on discrete state-spaces: a synthesis , (2020) by Lancelot Da Costa and Thomas Parr and Noor Sajid and Sebastijan Veselic and Victorita Neacsu and Karl Friston [bib]
This is a great and thorough tutorial on discrete-state-space active inference. I would reccomend it to everybody new to the field.
- Active inference and epistemic value , (2015) by Karl Friston and Francesco Rigoli and Dimitri Ognibene and Christoph Mathys and Thomas Fitzgerald and Giovanni Pezzulo [bib]
Introduces the main intuitions behind active inference, as well as the crucial epistemic foraging behaviour of the expected free energy. Illustrated on a simple T-maze task.
- Active inference and learning , (2016) by Karl Friston and Thomas FitzGerald and Francesco Rigoli and Philipp Schwartenbeck and Giovanni Pezzulo and others [bib]
- Active inference and agency: optimal control without cost functions , (2012) by Karl Friston and Spyridon Samothrakis and Read Montague [bib]
The first (I think) discrete-state-space paper on active inference. Notable for using the standard variational free energy as objective function and not the expected free energy. Describes some of the intuitions behind active inference.
- Active inference: a process theory , (2017) by Karl Friston and Thomas FitzGerald and Francesco Rigoli and Philipp Schwartenbeck and Giovanni Pezzulo [bib]
Provides a very good and thorough description of discrete-state-space active inference and ties its updates closely to neural physiology. I would reccomend this after the Da Costa introduction.
- Uncertainty, epistemics and active inference , (2017) by Thomas Parr and Karl J Friston [bib]
- Deep temporal models and active inference , (2018) by Karl J Friston and Richard Rosch and Thomas Parr and Cathy Price and Howard Bowman [bib]
- Sophisticated Inference , (2020) by Karl Friston and Lancelot Da Costa and Danijar Hafner and Casper Hesp and Thomas Parr [bib]
Introduces the next stage of active inference. 'Sophisticated' active inference, where agents make decisions not just on their beliefs about the future, but on how their beliefs will change in the future. Allows the simulation of real epistemic value -- i.e. act so as to change your beliefs in the future.
- Active inference: demystified and compared , (2019) by Noor Sajid and Philip J Ball and Karl J Friston [bib]
- The relationship between dynamic programming and active inference: The discrete, finite-horizon case , (2020) by Lancelot Da Costa and Noor Sajid and Thomas Parr and Karl Friston and Ryan Smith [bib]
Discusses the relationship between active inference and dynamic programming solutions to reinforcement learning problems (i.e. Q learning, value functions etc). Shows that they are largely equivalent except with different objectives (Expected Free Energy vs Expected Discounted Reward).
- Reinforcement learning or active inference? , (2009) by Karl J Friston and Jean Daunizeau and Stefan J Kiebel [bib]
The earliest paper (I think) on active inference. Introduces the motivation behind the continuous state and time formulation of active inference. Shows how predictive coding can be used to learn actions as well as observations (by treating them the same)
- An active inference implementation of phototaxis , (2017) by Manuel Baltieri and Christopher L Buckley [bib]
Active inference in plants!!!
- PID control as a process of active inference with linear generative models , (2019) by Manuel Baltieri and Christopher L Buckley [bib]
Active inference under a linear gaussian generative model can replicate PID, but also provide a natural method for learning the tuning coefficients (by understanding them as precisions).
- On Kalman-Bucy filters, linear quadratic control and active inference , (2020) by Manuel Baltieri and Christopher L Buckley [bib]
A key step towards understanding how active inference relates to classical control theory methods such as Kalman Filters and LQR control.
- Application of the Free Energy Principle to Estimation and Control , (2019) by Thijs van de Laar and Ay{\c{c}}a {"O}z{\c{c}}elikkale and Henk Wymeersch [bib]
Another approach to understanding how active inference relates to and extends classical control theory methods.
- The State Space Formulation of Active Inference: Towards Brain-Inspired Robot Control , (2019) by Sherin Grimbergen [bib]
An excellent overview and fantastic piece of work on the linear time-indepenent formulation of active inference and its relation to classical control theory.
- Hierarchical active inference: A theory of motivated control , (2018) by Giovanni Pezzulo and Francesco Rigoli and Karl J Friston [bib]
- The graphical brain: belief propagation and active inference , (2017) by Karl J Friston and Thomas Parr and Bert de Vries [bib]
Introduces the general factor-graph message passing viewpoint on active inference. Also introduces hierarchical active inference models.
- Neuronal message passing using Mean-field, Bethe, and Marginal approximations , (2019) by Thomas Parr and Dimitrije Markovic and Stefan J Kiebel and Karl J Friston [bib]
Discusses in depth the different potential message passing inference algorithms which can be used to implement active inference on factor graphs.
- Active inference, belief propagation, and the bethe approximation , (2018) by Sarah Schw{"o}bel and Stefan Kiebel and Dimitrije Markovi{'c} [bib]
Introduces the Bethe free energy, as a result of making the Bethe approximation instead of the mean-field variational assumption to derive the message passing algorithms.
- Generalised free energy and active inference , (2019) by Thomas Parr and Karl J Friston [bib]
- Whence the Expected Free Energy? , (2020) by Beren Millidge and Alexander Tschantz and Christopher L Buckley [bib]
Discusses whether we can derive the expected free energy objective function on principled ground from the FEP, and discusses different potential objective functions for active inference.
- On the Relationship Between Active Inference and Control as Inference , (2020) by Beren Millidge and Alexander Tschantz and Anil K Seth and Christopher L Buckley [bib]
Discusses the relationship between Active Inference and Control as Inference, a variational framework for understanding action selection which has emerged from RL.
- Active inference and robot control: a case study , (2016) by L{'e}o Pio-Lopez and Ange Nizard and Karl Friston and Giovanni Pezzulo [bib]
- Active inference body perception and action for humanoid robots , (2019) by Guillermo Oliver and Pablo Lanillos and Gordon Cheng [bib]
- End-to-end pixel-based deep active inference for body perception and action , (2019) by Cansu Sancaktar and Pablo Lanillos [bib]
- Active inference for robot control: A factor graph approach , (2019) by Mees Vanderbroeck and Mohamed Baioumy and Daan van der Lans and Rens de Rooij and Tiis van der Werf [bib]
- A novel adaptive controller for robot manipulators based on active inference , (2020) by Corrado Pezzato and Riccardo Ferrari and Carlos Hern{'a}ndez Corbato [bib]
- Adaptive robot body learning and estimation through predictive coding , (2018) by Pablo Lanillos and Gordon Cheng [bib]
- Recent advances in the application of predictive coding and active inference models within clinical neuroscience , (2020) by Ryan Smith and Paul Badcock and Karl J Friston [bib]
A comprehensive review of neuroscientific and computational psychiatry applications of the FEP and Active Inference.
- The Predictive Global Neuronal Workspace: A Formal Active Inference Model of Visual Consciousness , (2020) by Christopher J Whyte and Ryan Smith [bib]
- Neurocomputational mechanisms underlying emotional awareness: insights afforded by deep active inference and their potential clinical relevance , (2019) by Ryan Smith and Richard D Lane and Thomas Parr and Karl J Friston [bib]
- Simulating emotions: An active inference model of emotional state inference and emotion concept learning , (2019) by Ryan Smith and Thomas Parr and Karl J Friston [bib]
- The hierarchical basis of neurovisceral integration , (2017) by Ryan Smith and Julian F Thayer and Sahib S Khalsa and Richard D Lane [bib]
- Active inference in OpenAI gym: a paradigm for computational investigations into psychiatric illness , (2018) by Maell Cullen and Ben Davey and Karl J Friston and Rosalyn J Moran [bib]
- Reinforcement Learning through Active Inference , (2020) by Alexander Tschantz and Beren Millidge and Anil K Seth and Christopher L Buckley [bib]
Demonstrates that the exploration afforded by the Expected Free Energy Objective is useful in a deep reinforcement learning setting. Also maintains uncertainty through model ensembles applied in a model-based RL setting.
- Scaling active inference , (2020) by Alexander Tschantz and Manuel Baltieri and Anil K Seth and Christopher L Buckley [bib]
Implements Deep Active Inference in a model-based RL setting using explicit planning with a transition model.
- Deep active inference as variational policy gradients , (2020) by Beren Millidge [bib]
Implements deep active inference in a model-free policy gradient setting by amortising the learning of the expected-free-energy value function. Uses a transition model for the state-information gain term in the expected free energy.
- Deep active inference , (2018) by Kai Ueltzh{"o}ffer [bib]
The first paper to try combining active inference with deep neural networks. Demonstrates the importance of the exploratory terms of the EFE to solve the mountain-car problem.
- Deep active inference agents using Monte-Carlo methods , (2020) by Zafeirios Fountas and Noor Sajid and Pedro AM Mediano and Karl Friston [bib]
Many thanks to @conorheins, Tomasz Korbak, Ryan Smith, Mel Andrews, Casper Hesp, and Manuel Baltieri for their helpful suggestions.
To contribute, please make pull requests adding entries to the bibtex file.
The README file was generated from bibtex using the bibtex_to_md.py
file.
The keywords to use for each classification (Survey, Discrete-state-space etc) can be found at the bottom of the .py file.
This code and structure is heavily inspired by https://github.com/optimass/continual_learning_papers.