Keynote Talks

Abstract

As foundation models continue to scale, the dominant bottleneck is shifting from compute to memory—capacity, bandwidth, and data movement. This trend is especially acute for emerging agentic AI systems, where long contexts, persistent state, and multi-step reasoning place unprecedented pressure on the memory hierarchy. This talk argues that breaking the memory wall requires a coordinated effort across the entire stack, and offers a unified perspective spanning algorithms, systems, architecture, and silicon.

The talk first considers how memory demand can be reduced at its source, through algorithm- and system-level techniques that reshape how models store and move data during inference. It then turns to memory-centric accelerator design, showing how such techniques can be co-designed with the underlying hardware rather than treated in isolation. Finally, it looks toward silicon realization and the future of memory-centric accelerators purpose-built for foundation models and agent systems. The unifying thread—from memory compression to scheduling to memory-centric architecture and specialized chips—points toward a practical path for sustaining the continued scaling of foundation models.

Abstract

Agentic AI is moving out of demos and into daily use, creating enormous demand for efficient inference: higher throughput, lower latency, and better efficiency in both dollars and joules. Meeting these targets requires rethinking the full inference + tools stack, from the specialized silicon that runs the models, to the system software that compiles, schedules, and serves them at scale, to the model architectures that determine what must be computed in the first place. In this talk, we will examine these layers with an eye toward the next major advances in hardware architecture, and how systems and algorithms can be co-designed to fully exploit them. Large gains in inference efficiency will come not from isolated improvements, but from treating hardware, systems, and models as an integrated stack.

Abstract

We are often told that we are in a golden age for computer architecture, but this moment is something more profound: a golden age for the computer architect. Advances in AI are collapsing the traditional boundaries between architecture, software, and hardware implementation, enabling architects to move from idea to realization across the full stack. Yet, despite this promise, today’s AI systems still struggle to reason about architectural design due to missing abstractions and outdated tooling. In this talk, I will argue that the next era of innovation is not just about better hardware, but about redefining how we design systems: introducing new machine-understandable abstractions, building AI-native tools, and embracing system-level thinking in a world of increasingly specialized and heterogeneous architectures. As AI workloads diversify and hardware fragments, the role of the architect expands—from optimizing components to defining design spaces, composing systems, and shaping the software–hardware interface. This is not just a technological shift, but a redefinition of what it means to be a computer architect.

Abstract

AI agents are emerging as a dominant workload in a wide range of applications, promising to be the vehicle that delivers the promised benefits of AI to enterprises and consumers. Unlike conventional software or static inference, agentic workloads are dynamic and structurally complex. These agents are often directed graphs of compute and I/O operations that span multi-modal data input and conversion (e.g. speech to text), data processing and context gathering (e.g. privacy filtering, vector DB lookups), LLM inferences, tool calls, etc. To scale AI agent usage, we need efficient and scalable deployment and agent-serving infrastructure. Today, however, the vast majority of these workloads are deployed on homogeneous, high-end, single-vendor infrastructure, which can often be quite expensive and limits broad rollout.

To tackle this challenge, we present a system design for dynamic orchestration of AI agent workloads on heterogeneous compute infrastructure spanning CPUs and accelerators, both from different vendors and across different performance tiers within a single vendor. The system delivers several building blocks: a framework for planning and optimizing agentic AI execution graphs using cost models that account for compute, memory, and bandwidth constraints of different HW; an MLIR-based compilation system that can decompose AI agent execution graphs into granular operators and generate code for different HW options; and a dynamic orchestration system that can place the granular components across a heterogeneous compute infrastructure and stitch them together while meeting an end-to-end SLA. Our design thus performs a system-level TCO optimization and our results show that leveraging a heterogeneous infrastructure can deliver significant TCO benefits.