This repository is a collection of publications related to probabilistic programming languages, probabilistic modelling, inference and criticism of probabilistic models.

1. Probabilistic Programming
2. Model Criticism
3. Probabilistic Modelling
4. Probabilistic Graphical Models

This section contains papers that are generally related to probabilistic programming and don't have a more specific subsection (yet).

• Freer2010 When are probabilistic programs probably computationally tractable?
• Ghahramani2015 Probabilistic machine learning and artificial intelligence
• Perov2015 Applications of Probabilistic Programming
• Perov2016 Automatic Sampler Discovery via Probabilistic Programming and Approximate Bayesian Computation
• Le2017 Inference Compilation and Universal Probabilistic Programming
• Mansinghka2018 Probabilistic Programming with Programmable Inference
• Baudart2018 Deep Probabilistic Programming Languages: A Qualitative Study
• CusumanoTower2018Incremental Inference for Probabilistic Programs
• Anikwue2019 Probabilistic Programming in Big Data
• Saad2019 Bayesian Synthesis of Probabilistic Programs for automatic Data Modeling

This section contains publications that introduce new languages or features for existing languages.

• Gilks1992 A language and program for complex Bayesian modelling
• Sheu1998 Simulation-based Bayesian inference using BUGS
• Kulkarni1999 Picture: A Probabilistic Programming Language for Scene Perception
• Pfeffer2005 The Design and Implementation of IBAL: A General-Purpose Probabilistic Language
• DeRaedt2007 ProbLog: A probabilistic Prolog and its application in link discovery
• Laskey2007 MEBN: A language for first-order Bayesian knowledge bases
• Lunn2009 The Bugs Project: Evolution, Critique and Future Directions
• Hershey2012 Accelerating Inference: towards a full Language,Compiler and Hardware stack (Dimple)
• Goodman2012 Church: A Language for generative Models
• Mansingkha2014 Venture: a higher-order probabilistic programming platform with programmable inference
• Gaunt2016 TerpreT: A Probabilistic Programming Language for Program Induction
• Tolpin2016 Design and Implementation of Probabilistic Programming Language Anglican
• Carpenter2017 Stan: A Probabilistic Programming Language
• Dillon2017 Tensorflow Distributions
• Tran2017 Deep Probabilistic Programming (Edward)
• DeValpine2017 Programming With Models: Writing Statistical Algorithms for General Model Structures With NIMBLE
• Ge2018 Turing: a language for flexible probabilistic inference
• Binfham2018 Pyro: Deep Universal Probabilistic Programming
• CusumanoTower2019 Gen: a general-purpose probabilistic programming system with programmable inference
• Piponi2020 Joint Distributions for TensorFlow Probability

Here we collect publications that apply probabilistic programming languages in active research.

• Zhang2015 Mixed Logical Inference and Probabilistic Planning for Robots in Unreliable Worlds
• Jacobs2016 Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial (STAN)
• Greiner2016 On The Fermi-GBM Event 0.4 s after GW150914 (PyMC3)
• Becker2017 Therapeutic reduction of ataxin-2 extends lifespan and reduces pathology in TDP-43 mice (STAN)
• Miller2017 Dorsal hippocampus contributes to model-based planning (STAN)
• Svenson2017 Power analysis of single-cell RNA-sequencing experiments (STAN)
• Kucukelbir2017 Automatic Differentiation Variational Inference (STAN)
• Graham2018 Seabirds Enhance Coral Reef Productivity (PyMC3)
• Baydin2019 Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
• Dehning2020 Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions (PyMC3) (Talk at PyMCon 2020)
• Brauner2020 The effectiveness of eight nonpharmaceutical interventions against COVID-19 in 41 countries (PyMC3)
• Papers using Infer.net

This section collects books or longer publications that focus primarily on probabilistic programming languages.

• VanDeMeent2018 An Introduction to Probabilistic Programming
• Pilon2015 Probabilistic Programming and Bayesian Methods for Hackers
• Roy2011 Computability, inference and modeling inprobabilistic programming

Here we collect media like talks and podcasts apart from official publications.

• David M. Blei - Black Box Variational Inference
• Michael Betancourt - Some Bayesian Modeling Techniques in Stan
• Michael Betancourt - Scalable Bayesian Inference with Hamiltonian Monte Carlo
• Andrew Gelman - Introduction to Bayesian Data Analysis and Stan
• Frank Wood - Inference Compilation
• Probprog Conference
• StanCon
• PyMC Developers / PyMCon

• The Talking Machines ANGLICAN and Probabilistic Programming
• The Talking Machines Probabilistic Programming and Digital Humanities

This section contains everything related to model criticism, inference diagnosis and everything that is about the assessment of model quality.

• Kass1995 Bayes Factors
• Ohagan2003 HSSS Model Criticism
• Krnjajic2008 Parametric and nonparametric Bayesian model specification: A casestudy involving models for count data
• Bayarri2007 Bayesian Checking of the Second Levels of Hierarchical Models
• Bayarri2007 A Framework for Validation of Computer Models
• Blei2014b Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models
• Lloyd2015 Statistical Model Criticism using Kernel Two Sample Tests
• Seth2018 Model Criticism in latent space
• Vehtari2019 Rank-normalization, folding, and localization: An improved Rˆ for assessing convergence of MCMC

• Gelman1997 Weak convergence and optimal scaling of random walk Metropolis algorithms

• Gelman1992 Inference from Iterative Simulation Using Multiple Sequences
• Brooks1998 General Methods for Monitoring Convergence of Iterative Simulations
• [Gelman2003] Bayesian Data Analysis, second edition
• Gelman2013 Bayesian Data Analysis, third edition
• Vehtari2020 Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC
• Lambert2020 R*: A robust MCMC convergence diagnostic with uncertainty using gradient-boosted machines Feel free to complete and/or correct any of these lists. Pull requests are very welcome! culty.ucr.edu/~jflegal/Final_Thesis_twosided.pdf) Monte Carlo Standard Errors for Markov Chain

• Akaike1973 Information Theory and an Extension of the Maximum Likelihood Principle
• Stone1977 An asymptotic equivalence of choice of model cross-validation and Akaike’s criterion
• Vehtari2002 Bayesian Model Assessment and Comparison UsingCross-Validation Predictive Densities
• Spiegelhalter2002 Bayesian measures of model complexity and fit
• Watanabe2010a Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• Vehtari2012 A survey of Bayesian predictive methods for model assessment, selection and comparison.
• Watanabe2013 A widely applicable Bayesian information criterion
• Gelman2013 Understanding predictive information criteria for Bayesian models
• Vehtari2015 Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC

• Korb2010 Bayesian Artificial Intelligence (Ch. 10)
• Coupe2000 Sensitivity Analysis of Decision-Theoretic Networks

• Guttman1967 The Use of the Concept of a Future Observation in Goodness‐Of‐Fit Problems
• Meng1994 Posterior predictive p-values
• Gelman1996 Posterior Predictive Assessment of Model Fitness Via Realized Discrepancies
• Lewis1996 Comment on ‘Posterior predictive assessment of model fitness via realized discrepancies’
• Hoijtink1997 A multidimensional item response model: constrained latent class analysis using the Gibbs sampler and posterior predictive checks.
• Berkhoff2000 Posterior predictive checks: Principles and Discussion
• Gelman2002 Diagnostic checks for discrete-data regression models using posterior predictive simulations.
• Gelman2007 Data Analysis using Regression and Multilevel/Hierarchical Models
• Gelman2009
• Kruschke2015 Bayesian estimation supersedes the t test.
• Gabry2019 Visualization in Bayesian workflow (loo-pit-ppc)

• Gneiting2007 Strictly Proper Scoring Rules, Prediction, and Estimation
• Cowell1993 Sequential Model Criticism in Probabilistic Expert Systems

• Klejnen1995 Statistical validation of simulation models
• Thacker2004 Concepts of Model Verification and Validation
• Sargent2011 Verification and Validation of Simulation Models
• Sargent2015 Use of the Interval Statistical Procedure for Simulation Model Validation
• Tsioptsias2016 Model Validation and Testing in Simulation: a Literature Review

• Gelfand1992 Model Determination Using Predictive Distributions With Implementation Via Sampling-Based Methods
• Collins2014 External Validation of Multivariable Prediction Models: A Systematic Review of Methodological Conduct and Reporting

• Chubarian2020 Interpretability of Bayesian Network Classifiers: OBDD Approximation and Polynomial Threshold Functions

• Shih2018 A Symbolic Approach to Explaining Bayesian Network Classifiers
• Timmer2017 A two-phase method for extracting explanatory arguments from Bayesian networks

This section contains publications that use visualization for model criticism.

• Kruschke2015 Bayesian Estimation in Hierarchical Models (Kruschke-style diagrams)
• Gabry2019 Visualization in Bayesian workflow

This section lists frameworks that provide model criticism functionality.

• Kumar2019 ArviZ is a unified library for exploratory analysis of Bayesian models in Python
• Bayesplot is an R package providing an extensive library of plotting functions for use after fitting Bayesian models (typically with MCMC)

Here we collect media like talks and podcasts apart from official publications.

• Oriol Unraveled

• Aki Vehtari - These are a few of my favorite inference diagnostics
• Rob Zinkov - A Tour of Model Checking techniques

• Gelman2002 Prior Distribution
• Gelman2004 Parameterization and Bayesian Modelling
• Skrondal2007 Latent Variable Modelling: A Survey
• Gelman2009 Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics
• Kass2012 The Selection of Prior Distributions by Formal Rules/
• Kruschke2015 Bayesian Estimation in Hierarchical Models (Kruschke-style diagrams)
• Gelman2017 The Prior can generally only be understood in the Context of the Likelihood

• Gelman2013 Bayesian Data Analysis
• McElreath2015 Statistical Rethinking

Here we collect media like talks and podcasts apart from official publications.

• Modern Statistical Workflow
• Robust Statistical Workflow with RStan

• Pearl1988 Probabilistic Reasoning in Intelligent Reasoning
• Pear2000 Bayesian Networks
• Pitchforth2013 Expert Systems with Applications
• Frey2003 Extending Factor Graphs so as to Unify Graphical Models
• Jordan2004 Graphical Models

• Awesome SPN

• Wainright2008 Graphical Models, Exponential Families, and Variational Inference
• Koller2009 Probabilistic Graphical Models: Principles and Techniques
• Cowel2006 Probabilistic Networks and Expert Systems
• Bishop2006 Pattern Recognition and Machine Learning

• Romero2009 Triangulation of Bayesian networks with recursive estimation of distribution algorithms
• Peyrard2018 Exact or approximate inference in graphical models: why the choice is dictated by the treewidth, and how variable elimination can be exploited

This section contains publications that focus on or involve exact inference.

• Pearl1998 Probabilistic Reasoning in Intelligent Systems
• Copper1990 The Computational Complexity of Probabilistic Inference using Bayesian Belief Networks

• Lauritzen1988 Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems
• Jensen1990 Bayesian updating in Causal Probabilistic Networks by local Computation

• Wellman2013 State-space Abstraction for Anytime Evaluation of Probabilistic Networks
• Dagum1993 Approximating probabilistic inference in Bayesian belief networks is NP-hard
• Neil2003 Slice Sampling
• Andrieu2010 Particle Markov chain Monte Carlo methods
• DuBois2014 Approximate Slice Sampling for Bayesian Posterior Inference
• Paige2014 Asynchronous Anytime Sequential Monte Carlo
• Naeseth2016 High-dimensional Filtering using Nested Sequential Monte Carlo
• Crisan2017 Nested particle filters for online parameter estimation in discrete–time state–space Markov models

• Ng2000 Approximate Inference Algorithms for two-layer Bayesian Networks

• Agapiou2017 Importance Sampling: Intrinsic Dimension and Computational Cost

• Paige2016 Inference Networks for Sequential Monte Carlo in Graphical Models
• Le2017 Inference Compilation and Universal Probabilistic Programming

• Metropolis1953 Equation of State Calculations by Fast Computing Machines
• Robert2016 The Metropolis–Hastings Algorithm

• Gilks1992 Adaptive Rejection Sampling for Gibbs Sampling
• Jensen1995 Blocking Gibbs sampling in very large probabilistic expert systems
• Gelfand2000 Gibbs Sampling

• Duane1987 Hybrid Monte Carlo
• Arouna2004 Adaptive Monte Carlo Method, A Variance Reduction Technique
• Hoffman2011 The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Betancourt2017 A conceptual introduction to Hamiltonian Monte Carlo

• Jordan1999 An Introduction to Variational Methods for Graphical Models
• Jaakkola1999 Variational Probabilistic Inference and the QMR-DT Network
• Blei2018 Variational Inference: A Review for Statisticians
• Kucukelbir2017 Automatic Differentiation Variational Inference

• Robert2000 Monte Carlo Statistical Method
• Geyer2011 Introduction to Markov Chain Monte Carlo

• Richard McElreath's Blog Markov Chains: Why Walk When You Can Flow?