"Introduction to Machine Programming"

Course Description

The field of machine programming (MP) is concerned with the automation of software development. Given recent advances in algorithms, hardware efficiency and capacity, and an ever increasing availability of code data, it is now possible to train machines to help develop software. In this course, we teach students how to build real-world MP systems. We begin with a high-level overview of the field, including an abbreviated analysis of state-of-the-art MP systems (e.g., DeepMind's AlphaCode, GitHub's Copilot, Merly's Mentor). Next, we discuss the foundations of MP and the key areas for innovation, some of which are unique to MP. We close with a discussion of current limitations and future possibilities of MP. This course includes a nine-week hands-on project, where students (as individuals or in a small group) will create their own MP system and demonstrate it to the class.

While some overlap exists between traditional techniques to train machines to perform non-programming tasks (e.g., natural language processing, computer vision, etc.), teaching machines to perform programming-specific tasks has uniqueness in at least two dimensions. First, there are certain techniques that are more (or less) effective for MP, such as using self-supervision to learn from the large corpora of unlabeled open-source code. Second, software reasoning is fundamentally multi-dimensional; that is, there exist multiple unique ways to learn from software (e.g., static analysis, dynamic analysis, input/output specifications, program state reinforced-convergence, hardware telemetric data, etc.). In this course, we discuss each of these techniques (and others) and how they can be effectively applied to MP systems.

The course lectures cover the following major segments:

Overview: two lectures: lecture 1 focuses on the foundations of MP and some background necessary to understand the field, lecture 2 briefly covers the current state-of-the-art MP systems.

Learning: two lectures: lecture 1 presents the fundamentals of machine learning and classical learning techniques (e.g, supervised learning with neural networks), lecture 2 covers emerging techniques necessary for MP-specific learning (e.g., self-supervised static analysis, dynamic program analysis and data generation, etc.).

Three Pillars of MP: three lectures spread evenly across the three pillars of MP: intention, invention, and adaptation; each lecture also includes some background material to bring students up to speed on domain-specific details in the areas of programming languages, code abstractions and representations, software engineering principles, and compilers, amongst others.

Deep Data: three lectures about the various ways to reason about, generate, and utilize data for MP systems: lecture 1 discusses classical ML-based data utilization (e.g., training, validation, and testing), lecture 2 covers future ways to harness and automatically synthesize and label data for MP systems (e.g., program synthesis for data generation, dynamic execution information for analysis, automated semantics labeling, etc.).

Semantic Reasoning: two lectures on the foundations, construction, and utilization of semantic reasoning systems in MP: lecture 1 covers some of the basics found in other CS courses that are necessary for advanced reasoning about syntax and semantics, lecture 2 covers details on how to construct semantics representations and use them for downstream MP tasks.