Pyro

Pyro 是什么？

Pyro is a deep probabilistic programming language(PPL) released by Uber AI Labs. Pyro is built on top of PyTorch and is based on four fundamental principles:

• Universal: Pyro is a universal PPL — it can represent any computable probability distribution. How? By starting from a universal language with iteration and recursion (arbitrary Python code), and then adding random sampling, observation, and inference.
• Scalable: Pyro scales to large data sets with little overhead above hand-written code. How? By building modern black box optimization techniques, which use mini-batches of data, to approximate inference.
• Minimal: Pyro is agile and maintainable. How? Pyro is implemented with a small core of powerful, composable abstractions. Wherever possible, the heavy lifting is delegated to PyTorch and other libraries.
• Flexible: Pyro aims for automation when you want it and control when you need it. How? Pyro uses high-level abstractions to express generative and inference models, while allowing experts to easily customize inference.

See more details on blog!

Powerful tool for model-based causal inference

In addition to pyro.condition for incorporating observations, Pyro also contains pyro.do, an implementation of Pearl's do-operator used for causal inference with an identical interface to pyro.condition. condition and do can be mixed and composed freely, making Pyro a powerful tool for model-based causal inference.

灵魂拷问

• Why probabilistic modeling? To correctly capture uncertainty in models and predictions for unsupervised and semi-supervised learning, and to provide AI systems with declarative prior knowledge.
• Why (universal) probabilistic programs? To provide a clear and high-level, but complete, language for specifying complex models.
• Why deep probabilistic models? To learn generative knowledge from data and reify knowledge of how to do inference.
• Why inference by optimization? To enable scaling to large data and leverage advances in modern optimization and variational inference.
• 为什么学习 Pyro? 从数学本质上来书类似于从一般的数到随机变量的推广，类似于从面向过程到面向对象的推广，在 Pyro 中一切都是随机变量，它提供了对数据建模的更强大的表达工具。Probability is the mathematics of reasoning under uncertainty, much as calculus is the mathematics for reasoning about rates of change. Models built in the language of probability can capture complex reasoning, know what they do not know, and uncover structure in data without supervision. Further, probability provides a way for human experts to provide knowledge to AI systems in the form of a priori beliefs.

对比其他概率图模型工具：

• Pyro vs Edward: like Edward, Pyro is a deep probabilistic programming language that focuses on variational inference but supports general composable inference algorithms. Pyro aims to be more dynamic (by using PyTorch) and universal (allowing recursion). In particular it should be easy to integrate existing simulator libraries into Pyro as long as they have a Python interface. The Pyro, Edward, and Probabilistic Torch 179 teams are collaborating to identify common concepts and to ensure models are maximally compatible across our systems.
• Pyro vs PyMC, Stan: Pyro embraces deep neural nets and currently focuses on variational inference. Pyro doesn’t do MCMC yet (but we’re actively working on HMC). Whereas Stan models are written in the Stan language, Pyro models are just python programs with pyro.sample() statements. Stan caters to the stats community who value interpretability and model criticism. Pyro caters more to the ML community, and tries to add some statistical tools to existing PyTorch models.
• Pyro vs Gen: Based on the Julia programming language. MIT unveiled a new probabilistic programming language named Gen. The new language allow researchers to write models and algorithms from multiple fields where AI techniques are applied — such as computer vision, robotics, and statistics — without having to deal with equations or manually write high-performance code. Pyro 是强有力的竞争对手。

FAQ

1. Is Pyro worth learning?

Absolutely!!!!

2. What is PPL?

The field of probabilistic programming languages(PPL) is experiencing a marvelous renaissance carried by the rapid growth of machine learning technologies. PPLs are a regular component of machine learning pipelines but its implementation remains challenging. The combination of statistics and deep learning is a key element of the future of the artificial intelligence space.