The Computer Science of Accelerator Design

Underlying every computing system, from the smallest embedded sensor to the largest warehouse-scale distributed system, ultimately is some form of computing hardware. All software abstractions—from the application logic, to language run-time, operating system, and virtual machine—in the end perform their function through a set of low-level commands to a physical device. However, increasingly the underlying hardware is not a general purpose machine but instead a collection of application-focused accelerators developed to provide better performance under a given energy and area budget. The design of these accelerators is not the domain of Electrical Engineers alone -- there there is a tremendous amount of Computer Science to be done as well.

From Microsoft’s Catapult, to Amazon’s F1 service, to Google’s ASIC cloud, it is clear that to achieve the energy efficiency demanded by computing at enormous scale we will need specialized machines for everything from neural network training and inference to media re-coding and beyond. The design of such accelerators will require new algorithms more suited to efficient hardware operation, new programming languages that can cleanly and rapidly define computations ready for acceleration, and new software tools for exploring and rapidly prototyping design options. How can the abstract algebra of semirings help us think about accelerator design? What does a monad in hardware look like? How is accelerator design increasingly a software engineering problem? Studying papers from Google Brain, Microsoft Research, and leading academics, we will explore potential answers to these questions, and students will leave the course with a better understanding of what things are efficient in hardware versus software, how to reason about the size, delay, and energy efficiency of neural network accelerator designs, and what exactly this next generation of languages for hardware design are doing under the hood. The course will have a series of smaller projects that will help establish the framework for taking on a larger accelerator research effort of the student’s own design.

Expectations

To get the most from this class one should have some background in the basics of computer architecture (including the concepts of caches, branch predictors, assembly language, and elementary digital design), a working understanding of automata theory (including finite state machines, NFAs, regular expressions, and grammars). Background in functional programming, hardware design, and advanced software engineering concepts will be helpful but is certainly not necessary. As students in a graduate course, when papers touch upon concepts that one is not familiar with the expectation is that students will take that as an invitation to learn the background necessary to fully understand the work described -- if suggestions of background material is needed I am happy to help point you in a helpful direction. I don't expect any student will have a strong background in all the areas covered because even I don't! However, students are expected to come to class having prepared by reading the papers required and be really to participate constructively in discussions.