Pyro 是什么？
Pyro is a deep probabilistic programming language(PPL) released by Uber AI Labs. Pyro is built on top of PyTorch and is based on four fundamental principles:
- Universal: Pyro is a universal PPL — it can represent any computable probability distribution. How? By starting from a universal language with iteration and recursion (arbitrary Python code), and then adding random sampling, observation, and inference.
- Scalable: Pyro scales to large data sets with little overhead above hand-written code. How? By building modern black box optimization techniques, which use mini-batches of data, to approximate inference.
- Minimal: Pyro is agile and maintainable. How? Pyro is implemented with a small core of powerful, composable abstractions. Wherever possible, the heavy lifting is delegated to PyTorch and other libraries.
- Flexible: Pyro aims for automation when you want it and control when you need it. How? Pyro uses high-level abstractions to express generative and inference models, while allowing experts to easily customize inference.
Powerful tool for model-based causal inference
In addition to pyro.condition for incorporating observations, Pyro also contains pyro.do, an implementation of Pearl's do-operator used for causal inference with an identical interface to pyro.condition. condition and do can be mixed and composed freely, making Pyro a powerful tool for model-based causal inference.
Universal: Pyro can represent any computable probability distribution.
Scalable: Pyro scales to large data sets with little overhead.
Minimal: Pyro is implemented with a small core of powerful, composable abstractions.
Flexible: Pyro aims for automation when you want it, control when you need it.
Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. Pyro enables flexible and expressive deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling. It was designed with these key principles: Universal, Scalable, Flexible, Minimal
Pyro is a deep probabilistic programming language built on PyTorch, a GPU-accelerated deep learning framework. Developed at Uber AI Labs by Noah Goodman and team, Pyro is used as a platform for research in modern Bayesian machine learning, where deep neural networks can be used both in models and in inference. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch. The Pyro team works closely with the PyTorch team and many open source collaborators to create a rich, stable tool set for probabilistic machine learning research.
References:
[2] Gen: a general-purpose probabilistic programming system with programmable inference (Marco F. Cusumano, 2019)
灵魂拷问
- Why probabilistic modeling? To correctly capture uncertainty in models and predictions for unsupervised and semi-supervised learning, and to provide AI systems with declarative prior knowledge.
- Why (universal) probabilistic programs? To provide a clear and high-level, but complete, language for specifying complex models.
- Why deep probabilistic models? To learn generative knowledge from data and reify knowledge of how to do inference.
- Why inference by optimization? To enable scaling to large data and leverage advances in modern optimization and variational inference.
- 为什么学习 Pyro? 从数学本质上来书类似于从一般的数到随机变量的推广，类似于从面向过程到面向对象的推广，在 Pyro 中一切都是随机变量，它提供了对数据建模的更强大的表达工具。Probability is the mathematics of reasoning under uncertainty, much as calculus is the mathematics for reasoning about rates of change. Models built in the language of probability can capture complex reasoning, know what they do not know, and uncover structure in data without supervision. Further, probability provides a way for human experts to provide knowledge to AI systems in the form of a priori beliefs.
对比其他概率图模型工具：
- Pyro vs Edward: like Edward, Pyro is a deep probabilistic programming language that focuses on variational inference but supports general composable inference algorithms. Pyro aims to be more dynamic (by using PyTorch) and universal (allowing recursion). In particular it should be easy to integrate existing simulator libraries into Pyro as long as they have a Python interface. The Pyro, Edward, and Probabilistic Torch 179 teams are collaborating to identify common concepts and to ensure models are maximally compatible across our systems.
- Pyro vs PyMC, Stan: Pyro embraces deep neural nets and currently focuses on variational inference. Pyro doesn’t do MCMC yet (but we’re actively working on HMC). Whereas Stan models are written in the Stan language, Pyro models are just python programs with pyro.sample() statements. Stan caters to the stats community who value interpretability and model criticism. Pyro caters more to the ML community, and tries to add some statistical tools to existing PyTorch models.
- Pyro vs Gen: Based on the Julia programming language. MIT unveiled a new probabilistic programming language named Gen. The new language allow researchers to write models and algorithms from multiple fields where AI techniques are applied — such as computer vision, robotics, and statistics — without having to deal with equations or manually write high-performance code. Pyro 是强有力的竞争对手。
介绍视频 Videos
Pyro enables flexible and expressive deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling.
概率编程 <-- History of Programming <-- Deep Learning 时代有特点 <-- Diff between Prob & Deterministic Programing
- 概率模型比 deterministic 模型好
- 需要对生成数据过程编程和使用
why create it?
- probabilistic models
- high-level programming languages
- scalable deep learning
Probabilistic Modeling in AI: Frontiers, Goals for pyro.
概率编程
MLTrain@UAI2018, Pyro
- 结合了深度学习和贝叶斯模型
- 通用概率编程语言
- 可以包含随机控制结构
FAQ
1. Is Pyro worth learning?
Absolutely!!!!
2. What is PPL?
The field of probabilistic programming languages(PPL) is experiencing a marvelous renaissance carried by the rapid growth of machine learning technologies. PPLs are a regular component of machine learning pipelines but its implementation remains challenging. The combination of statistics and deep learning is a key element of the future of the artificial intelligence space.