Probabilistic Programming

Probabilistic programming is an idea of describing probabilistic models as a kind of computer programs. A typical probabilistic program describes a generative process where latent and observable random variables and their relationships are specified; formally, a probabilistic program defines a so-called unnormalized posterior for the latent random variables. Given a probabilistic program, one's goal is to find quantities relevant to the program: one is (precise or approximate) computation of the (normalized) posterior for the latent variables; another is learning useful model parameters, and there is more.

Probabilistic programming has potentials for broader scientific and engineering impacts. One contributory factor is the expressive power of a universal probabilistic programming language (PPL) and formal techniques from the programming-languages community that may be applied to such a PPL. Another is recent advances (oftentimes under the name of deep probabilistic programming) in the marriage between deep learning and probabilistic reasoning.

Probabilistic programming is not just an idea; it comes with software systems that implement the ideas. When one "runs" a probabilistic program, the system does the target computation under the hood, and scientists and engineers focus on their domain applications rather than the computational details. The runtime support is one of the main selling points of probabilistic programming.