Learning from All Types of Experiences:

A Unifying Machine Learning Perspective

1:00PM -5:00PM (PT), Sunday August 23, 2020

@ KDD 2020


In handling wide range of experiences ranging from data instances, knowledge, constraints, to rewards, adversaries, and lifelong interplay in an ever-growing spectrum of tasks, contemporary ML/AI research has resulted in thousands of models, learning paradigms, optimization algorithms, not mentioning countless approximation heuristics, tuning tricks, and black-box oracles, plus combinations of all above. While pushing the field forward rapidly, these results also make a comprehensive grasp of existing ML techniques more and more difficult, and make standardized, reusable, repeatable, reliable, and explainable practice and further development of ML/AI products quite costly, if possible, at all.

This tutorial will present a systematic, unified perspective of machine learning, for both a refreshing holistic understanding of the diverse learning algorithms and a guidance of operationalizing machine learning for creating problem solutions by integrating all sources of experiences.

The tutorial consists of three parts: (1) Theory: a systematic blueprint of ML that provides a unified mathematical formulation for learning with all experiences; (2) Tooling that operationalizes the framework and enables easy composition of ML solutions; (3) Computing infrastructures for productive ML, including interoperation, automatic tuning, distributing, and scheduling.

Schedule and materials

  • Introduction (10min)

  • Part I: Theory: The Standard Equation (90min)

A blueprint of ML paradigms for ALL experiences

    • Background

    • The Standard Equation

    • The zoo of optimization solvers

    • Applications of Standard Equation

  • Part II: Tooling: Operationalizing “Learning with all experiences” (70min)

Compose your ML solutions like playing Lego

    • ML solution design by learning with all experiences

    • Compositionality in ML

    • Texar: an open-source tool for ML Composition

  • Part III: Computing: Modern infrastructure for productive ML (40min)

Interoperation, automatic tuning, distributing, and scheduling

  • Interoperation: Forte

  • Automatic tuning: TUUN

  • Automatic distributing: AutoDist

  • Automatic scheduling: AdaptDL


Zhiting Hu

PhD @ Carnegie Mellon Univ.

Incoming Assist. Prof. @ UC San Diego

Qirong Ho

CTO @ Petuum Inc.

Eric P. Xing

Professor @ Carnegie Mellon Univ.

Co-founder, Chief Scientist @ Petuum Inc.