This repository is a collection of publications related to probabilistic programming languages, probabilistic modelling, inference and criticism of probabilistic models.
- You might enjoy the mindmap-view made possible by markmap.
- This collection is released under .
- Contribution is very welcome!
- It is inspired by the formidable awesome-spn.
- Probabilistic Programming
- Model Criticism
- Probabilistic Modelling
- Probabilistic Graphical Models
- Inference
This section contains papers that are generally related to probabilistic programming and don't have a more specific subsection (yet).
- Freer2010 When are probabilistic programs probably computationally tractable?
- Ghahramani2015 Probabilistic machine learning and artificial intelligence
- Perov2015 Applications of Probabilistic Programming
- Perov2016 Automatic Sampler Discovery via Probabilistic Programming and Approximate Bayesian Computation
- Le2017 Inference Compilation and Universal Probabilistic Programming
- Mansinghka2018 Probabilistic Programming with Programmable Inference
- Baudart2018 Deep Probabilistic Programming Languages: A Qualitative Study
- CusumanoTower2018Incremental Inference for Probabilistic Programs
- Anikwue2019 Probabilistic Programming in Big Data
- Saad2019 Bayesian Synthesis of Probabilistic Programs for automatic Data Modeling
This section contains publications that introduce new languages or features for existing languages.
- Gilks1992 A language and program for complex Bayesian modelling
- Sheu1998 Simulation-based Bayesian inference using BUGS
- Kulkarni1999 Picture: A Probabilistic Programming Language for Scene Perception
- Pfeffer2005 The Design and Implementation of IBAL: A General-Purpose Probabilistic Language
- DeRaedt2007 ProbLog: A probabilistic Prolog and its application in link discovery
- Laskey2007 MEBN: A language for first-order Bayesian knowledge bases
- Lunn2009 The Bugs Project: Evolution, Critique and Future Directions
- McCallum2009 FACTORIE: Probabilistic Programming via Imperatively Defined Factor Graphs
- Hershey2012 Accelerating Inference: towards a full Language,Compiler and Hardware stack (Dimple)
- Goodman2012 Church: A Language for generative Models
- Mansingkha2014 Venture: a higher-order probabilistic programming platform with programmable inference
- Gaunt2016 TerpreT: A Probabilistic Programming Language for Program Induction
- Tolpin2016 Design and Implementation of Probabilistic Programming Language Anglican
- Carpenter2017 Stan: A Probabilistic Programming Language
- Dillon2017 Tensorflow Distributions
- Tran2017 Deep Probabilistic Programming (Edward)
- DeValpine2017 Programming With Models: Writing Statistical Algorithms for General Model Structures With NIMBLE
- Ge2018 Turing: a language for flexible probabilistic inference
- Binfham2018 Pyro: Deep Universal Probabilistic Programming
- CusumanoTower2019 Gen: a general-purpose probabilistic programming system with programmable inference
- Piponi2020 Joint Distributions for TensorFlow Probability
Here we collect publications that apply probabilistic programming languages in active research.
- Zhang2015 Mixed Logical Inference and Probabilistic Planning for Robots in Unreliable Worlds
- Jacobs2016 Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial (STAN)
- Greiner2016 On The Fermi-GBM Event 0.4 s after GW150914 (PyMC3)
- Becker2017 Therapeutic reduction of ataxin-2 extends lifespan and reduces pathology in TDP-43 mice (STAN)
- Miller2017 Dorsal hippocampus contributes to model-based planning (STAN)
- Svenson2017 Power analysis of single-cell RNA-sequencing experiments (STAN)
- Kucukelbir2017 Automatic Differentiation Variational Inference (STAN)
- Graham2018 Seabirds Enhance Coral Reef Productivity (PyMC3)
- Baydin2019 Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
- Dehning2020 Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions (PyMC3) (Talk at PyMCon 2020)
- Brauner2020 The effectiveness of eight nonpharmaceutical interventions against COVID-19 in 41 countries (PyMC3)
- Papers using Infer.net
This section collects books or longer publications that focus primarily on probabilistic programming languages.
- VanDeMeent2018 An Introduction to Probabilistic Programming
- Pilon2015 Probabilistic Programming and Bayesian Methods for Hackers
- Roy2011 Computability, inference and modeling inprobabilistic programming
Here we collect media like talks and podcasts apart from official publications.
- David M. Blei - Black Box Variational Inference
- Michael Betancourt - Some Bayesian Modeling Techniques in Stan
- Michael Betancourt - Scalable Bayesian Inference with Hamiltonian Monte Carlo
- Andrew Gelman - Introduction to Bayesian Data Analysis and Stan
- Frank Wood - Inference Compilation
- Probprog Conference
- StanCon
- PyMC Developers / PyMCon
- The Talking Machines ANGLICAN and Probabilistic Programming
- The Talking Machines Probabilistic Programming and Digital Humanities
This section contains everything related to model criticism, inference diagnosis and everything that is about the assessment of model quality.
- Oreskes1994 Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences
- Kass1995 Bayes Factors
- Ohagan2003 HSSS Model Criticism
- Krnjajic2008 Parametric and nonparametric Bayesian model specification: A casestudy involving models for count data
- Bayarri2007 Bayesian Checking of the Second Levels of Hierarchical Models
- Bayarri2007 A Framework for Validation of Computer Models
- Blei2014b Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models
- Lloyd2015 Statistical Model Criticism using Kernel Two Sample Tests
- Seth2018 Model Criticism in latent space
- Vehtari2019 Rank-normalization, folding, and localization: An improved Rˆ for assessing convergence of MCMC
- Gelman1997 Weak convergence and optimal scaling of random walk Metropolis algorithms
- Gelman1992 Inference from Iterative Simulation Using Multiple Sequences
- Brooks1998 General Methods for Monitoring Convergence of Iterative Simulations
- [Gelman2003] Bayesian Data Analysis, second edition
- Gelman2013 Bayesian Data Analysis, third edition
- Vehtari2020 Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC
- Lambert2020 R*: A robust MCMC convergence diagnostic with uncertainty using gradient-boosted machines
- Vehtari2020 Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC
- Flegal2008 Monte Carlo Standard Errors for Markov Chain
- Akaike1973 Information Theory and an Extension of the Maximum Likelihood Principle
- Stone1977 An asymptotic equivalence of choice of model cross-validation and Akaike’s criterion
- Vehtari2002 Bayesian Model Assessment and Comparison UsingCross-Validation Predictive Densities
- Spiegelhalter2002 Bayesian measures of model complexity and fit
- Watanabe2010a Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
- Vehtari2012 A survey of Bayesian predictive methods for model assessment, selection and comparison.
- Watanabe2013 A widely applicable Bayesian information criterion
- Gelman2013 Understanding predictive information criteria for Bayesian models
- Vehtari2015 Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Korb2010 Bayesian Artificial Intelligence (Ch. 10)
- Coupe2000 Sensitivity Analysis of Decision-Theoretic Networks
- Guttman1967 The Use of the Concept of a Future Observation in Goodness‐Of‐Fit Problems
- Meng1994 Posterior predictive p-values
- Gelman1996 Posterior Predictive Assessment of Model Fitness Via Realized Discrepancies
- Lewis1996 Comment on ‘Posterior predictive assessment of model fitness via realized discrepancies’
- Hoijtink1997 A multidimensional item response model: constrained latent class analysis using the Gibbs sampler and posterior predictive checks.
- Berkhoff2000 Posterior predictive checks: Principles and Discussion
- Gelman2002 Diagnostic checks for discrete-data regression models using posterior predictive simulations.
- Gelman2007 Data Analysis using Regression and Multilevel/Hierarchical Models
- Gelman2009
- Kruschke2015 Bayesian estimation supersedes the t test.
- Gabry2019 Visualization in Bayesian workflow (loo-pit-ppc)
- Gneiting2007 Strictly Proper Scoring Rules, Prediction, and Estimation
- Cowell1993 Sequential Model Criticism in Probabilistic Expert Systems
- Klejnen1995 Statistical validation of simulation models
- Thacker2004 Concepts of Model Verification and Validation
- Sargent2011 Verification and Validation of Simulation Models
- Sargent2015 Use of the Interval Statistical Procedure for Simulation Model Validation
- Tsioptsias2016 Model Validation and Testing in Simulation: a Literature Review
- Gelfand1992 Model Determination Using Predictive Distributions With Implementation Via Sampling-Based Methods
- Collins2014 External Validation of Multivariable Prediction Models: A Systematic Review of Methodological Conduct and Reporting
- Chubarian2020 Interpretability of Bayesian Network Classifiers: OBDD Approximation and Polynomial Threshold Functions
- Shih2018 A Symbolic Approach to Explaining Bayesian Network Classifiers
- Timmer2017 A two-phase method for extracting explanatory arguments from Bayesian networks
This section contains publications that use visualization for model criticism.
- Kruschke2015 Bayesian Estimation in Hierarchical Models (Kruschke-style diagrams)
- Gabry2019 Visualization in Bayesian workflow
This section lists frameworks that provide model criticism functionality.
- Kumar2019 ArviZ is a unified library for exploratory analysis of Bayesian models in Python
- Bayesplot is an R package providing an extensive library of plotting functions for use after fitting Bayesian models (typically with MCMC)
Here we collect media like talks and podcasts apart from official publications.
- Aki Vehtari - These are a few of my favorite inference diagnostics
- Rob Zinkov - A Tour of Model Checking techniques
- Gelman2002 Prior Distribution
- Gelman2004 Parameterization and Bayesian Modelling
- Skrondal2007 Latent Variable Modelling: A Survey
- Gelman2009 Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics
- Kass2012 The Selection of Prior Distributions by Formal Rules/
- Kruschke2015 Bayesian Estimation in Hierarchical Models (Kruschke-style diagrams)
- Gelman2017 The Prior can generally only be understood in the Context of the Likelihood
- Gelman2013 Bayesian Data Analysis
- McElreath2015 Statistical Rethinking
Here we collect media like talks and podcasts apart from official publications.
- Kinderman1980 On the relation between Markov random fields and social networks
- Pearl1988 Probabilistic Reasoning in Intelligent Reasoning
- Pear2000 Bayesian Networks
- Pitchforth2013 Expert Systems with Applications
- Frey2003 Extending Factor Graphs so as to Unify Graphical Models
- Jordan2004 Graphical Models
- Cowel2006 Probabilistic Networks and Expert Systems
- Bishop2006 Pattern Recognition and Machine Learning
- Wainright2008 Graphical Models, Exponential Families, and Variational Inference
- Darwiche2009 Modeling and Reasoning with Bayesian Networks
- Koller2009 Probabilistic Graphical Models: Principles and Techniques
- Romero2009 Triangulation of Bayesian networks with recursive estimation of distribution algorithms
- Peyrard2018 Exact or approximate inference in graphical models: why the choice is dictated by the treewidth, and how variable elimination can be exploited
This section contains publications that focus on or involve exact inference.
- Pearl1998 Probabilistic Reasoning in Intelligent Systems
- Copper1990 The Computational Complexity of Probabilistic Inference using Bayesian Belief Networks
- Lauritzen1988 Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems
- Jensen1990 Bayesian updating in Causal Probabilistic Networks by local Computation
- Dagum1993 Approximating probabilistic inference in Bayesian belief networks is NP-hard
- Ng2000 Approximate Inference Algorithms for two-layer Bayesian Networks
- Neil2003 Slice Sampling
- Andrieu2010 Particle Markov chain Monte Carlo methods
- Wellman2013 State-space Abstraction for Anytime Evaluation of Probabilistic Networks
- DuBois2014 Approximate Slice Sampling for Bayesian Posterior Inference
- Paige2014 Asynchronous Anytime Sequential Monte Carlo
- Naeseth2016 High-dimensional Filtering using Nested Sequential Monte Carlo
- Crisan2017 Nested particle filters for online parameter estimation in discrete–time state–space Markov models
- Agapiou2017 Importance Sampling: Intrinsic Dimension and Computational Cost
- Paige2016 Inference Networks for Sequential Monte Carlo in Graphical Models
- Le2017 Inference Compilation and Universal Probabilistic Programming
- Metropolis1953 Equation of State Calculations by Fast Computing Machines
- Robert2016 The Metropolis–Hastings Algorithm
- Gilks1992 Adaptive Rejection Sampling for Gibbs Sampling
- Jensen1995 Blocking Gibbs sampling in very large probabilistic expert systems
- Gelfand2000 Gibbs Sampling
- Duane1987 Hybrid Monte Carlo
- Arouna2004 Adaptive Monte Carlo Method, A Variance Reduction Technique
- Hoffman2011 The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
- Betancourt2017 A conceptual introduction to Hamiltonian Monte Carlo
- Jordan1999 An Introduction to Variational Methods for Graphical Models
- Jaakkola1999 Variational Probabilistic Inference and the QMR-DT Network
- Blei2018 Variational Inference: A Review for Statisticians
- Kucukelbir2017 Automatic Differentiation Variational Inference
- Robert2000 Monte Carlo Statistical Method
- Geyer2011 Introduction to Markov Chain Monte Carlo
- Richard McElreath's Blog Markov Chains: Why Walk When You Can Flow?