Gemmini Tutorial at MLSys 2022

Gemmini is a platform for full-system, full-stack DNN accelerator evaluation. Gemmini allows users to generate a variety of different DNN hardware accelerators, with different underlying system, SoC, and programming stack components. 

Users can evaluate the performance of their hardware accelerators on end-to-end workloads in a real-world system context, exposing how different system components, like the cache hierarchy, virtual address translation scheme, or operating system, impact performance in subtle but noticeable ways. 

Gemmini also allows users to program their applications at different “levels” of the programming stack, from high-level model compilation to low-level direct machine configuration. 

Overall, Gemmini enables users to explore and evaluate a variety of different DNN accelerator and system configurations, exposing how these different parameters interact to impact end-to-end performance and efficiency.

1. Motivation and Architectural Template

   • Introduction

   • Architectural Template

     • SoC and System-Level Parameters

   • Interactive Activity: Build Your Own Gemmini

   • Interactive Activity: Add a New Datatype to Gemmini

2. Gemmini's Programming Model

   • Programming Stack

     • High-Level: ONNX Models

     • Mid-Level: Hand-Tuned Kernel Library

     • Low-Level: Direct Machine Configuration and C/Assembly

   • Interactive Activity: Visualize and Run ONNX Models

   • Interactive Activity: ResNet50 First-Layer Optimizations With Low-Level ISA

3. Performance Profiling on Gemmini

   • Types of Gemmini Counters

     • SoC Counters

     • Simulation Counters

   • Interactive Activity: Look at Simulation Logs

   • Interactive Activity: Evaluate Effect of TLB on DMA Performance

   • Interactive Activity: Use SoC Counters to Investigate Layer Bottlenecks

4. Parting Thoughts

5. Questions

This project was, in part, funded by the U.S. Government under the DARPA RTML program (contract FA8650-20-2-7006). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.