Program

Opening word

Hugo Pompougnac

Demistifying Different Attention Variants

Kunwar Grover (AMD)

Transformer based models have received unprecedented attention lately. One of the main bottlenecks for these models is the Attention layer, and can sometimes decide the entire execution time of these models. A number of recent research papers, have focused on different algorithms to codegen different Attention layer variants efficiently. The aim of this talk is to show how most of these proposed algorithms can be thought of as kernel fusions or known matrix multiplication optimizations. The talk will derive a base implementation for FlashAttention and use existing matrix multiplication like split-k as a model to derive different algorithms for attention variants. The talk will also cover how the MLIR-based compiler, IREE, codegens these kernels.

MLIR-based Data Tiling Design for Ryzen AI

Jorn Tuyls (AMD)

This workshop talk presents code generation for the NPU in Ryzen AI laptops and focuses mainly on the data tiling and data packing aspects of it. It shows how data tiling maps to the NPU's cores and caches and how it can describe granular data movement patterns through the NPU's data streams.

Hardware-Informed Domain-Specific Transformations

Sasha Lopoukhine (U. of Cambridge)

Towards AI model and architecture co-optimization for ultra-specialized hardware generation with Aidge

Olivier Bichler (CEA)

Aidge is a generic, multi-paradigms compute graph manipulation, quantization, mapping, scheduling and code generation tool. Its primary targets are embedded systems with specialized hardware accelerators, especially dataflow or restricted instruction set architectures. It is highly interoperable thanks to built-in ONNX import/export and direct PyTorch interface, and its modularity allows to use any of its features in standalone or in conjunction with other tools along the deployment path of a model to the embedded system. In this presentation, we will go over some of the main differentiating features of the framework and the contexts in which they may shine!

Formal Semantics as MLIR Dialects

Mathieu Fehr (U. of Edinburgh)

MLIR is designed to be modular and extensible, allowing for the definition of custom IRs. However, MLIR is primarily focused on syntax, and does not provide a way to define the semantics of operations in a formal way. This makes it difficult to reason about the correctness of transformations and analyses, and is a barrier to the development of formal verification tools for MLIR -based compilers. In this talk, we will introduce a set of semantics dialects, based on SMT -LIB, which allows to define the semantics of MLIR dialects as a compiler transformation. We will show how we can use these semantics dialects to give semantics of core MLIR dialects, such as arith, comb, and memref, and how we can use this new abstraction to define formal verification tooling such as a translation validation tool, a peephole rewrite verifier and synthetizer, and a dataflow analysis verifier.

Reactive programming is all you need

Dumitru Potop-Butucaru (INRIA KAIROS)

ML programming still involves today 3 different levels using different formalisms and practices: that of layers, that of models, and that of driver logic controlling the inference, training, or reinforcement learning process. Layers are typically linear algebra kernels, subject to classical HPC development methods. Layers and parameters are then assembled into models using increasingly complex control involving in current practice dataflow graphs, stateful behaviors, and conditional control. This is done using ML-specific formalisms such as jax or pytorch. Drivers are typically programmed in Python, and sometimes manually re-implemented in C/C++ for efficiency and/or embedding into larger applications. They take advantage of complex code transformations of the model, such as automatic differentiation, the synthesis of the parameter update code, or just-in-time compilation.

This split into 3 levels using different formalisms (and the use of little-formalized Python code) poses semantic problems, as one model will often behave differently on different architectures and under a different driver. But maybe more importantly, it limits automation and productivity. Full automation covers today the compilation of layers and models after transformations such as automatic differentiation and parameter update synthesis, as well as the automatic differentiation of stateless functions. However, the parameter update code, subject to subtle choices and working on a monolithic state representation, and the code interacting with the sample database or the I/Os must be hand-written.

In this presentation we will show that (dataflow) reactive languages provides a more natural programming paradigm encompassing layer, model, driver, and also the I/O processing and possibly the low-level scheduling of operations.

Reactive control primitives can be included as a dialect inside MLIR and then used jointly with tensor functions to allow modular, hierarchic, and stateful specification covering all three levels: layer, model and driver. The specification can then be seamlessly compiled into efficient code. Seen as a high-level specification language, the same reactive primitives support modular automatic differentiation and fully automatic synthesis of the parameter update code without the need to expose parameters at top level.

We would like to explore how these concepts can be further extended, leveraging  generic MLIR support for AD, code generation, and framework integration such as Enzyme-JAX.

MLIR is SSA + Regions + Dialects

Mehdi Amini

Interacting with MLIR/xDSL !

Mathieu Fehr & Sasha Lopoukhine

This session will include a hands-on tutorial to learn how to interact with xDSL to optimize and compile programs. While this tutorial will be focused on xDSL, the concepts and tools presented will be applicable to MLIR as well. We will showcase a few MLIR IR files, and show how to interpret them using the xdsl-opt tool. We will then present how to build a pipeline of passes to optimize these files and to generate low-level code.

Defining dialects & rewrites with xDSL

Mathieu Fehr & Sasha Lopoukhine

This session will be a hands-on tutorial on how to write new dialects and passes in xDSL. In particular, we will present how a pass composed of simple peephole rewrites (local optimizations) can be written in xDSL. We will task the participants with extending a high-level dialect with a new operation, and to then extend an optimization and transformation to a low-level dialect to support that new operation. While these tasks will be done in xDSL for simplicity, the concepts will be applicable to MLIR as well.

Defining dialects: ODS/Tablegen and C++

Mathieu Fehr

After learning the day before how to define operations, rewrites, and passes in xDSL, we will now present how to transfer this knowledge to MLIR. We will present how an MLIR dialect is structured in C++ and TableGen (a metaprogramming tool used by MLIR), and how to define new operations, attributes, and types. We will also present how to write new passes in C++, and how to write peephole rewrites using the MLIR pattern rewrite infrastructure.