Abstract:

Document parsing is often the first step in enterprise agentic workflows, from information extraction to retrieval on complex PDFs. At Databricks, we have been building custom document parsing models designed to be small, efficient, and optimized for chaining with other AI functions. In this talk, we'll discuss recent developments in our document processing models and highlight two cases where custom model development allows us to optimize for end-to-end task performance - from structured table lookup to multimodal reasoning.

Abstract:

The landscape of reinforcement learning (RL) for LLMs is evolving rapidly from paradigms like SFT/DPO and offline/online RLVR to OPD, each presenting trade-offs in ease of use, cost, and quality. To build Databricks-native agents, we developed an in-house Reinforcement Fine-Tuning (RFT) framework that consolidates these SOTA methods into a single, cohesive toolbox. We will discuss how our framework leverages SOTA algorithms and agentic harnessing to provide efficient OSS model training for Databricks agents, specifically for long-horizon tasks. Our results demonstrate that this framework delivers Pareto-optimal trained agents, enabling Databricks to achieve performance comparable to frontier proprietary providers at a fraction of the cost.

Abstract:

Enterprise assistants are entering a new phase: from single-turn question answering to fast, reliable knowledge work inside agentic workflows. This talk introduces Instructed-Retriever-1, the new retrieval model powering the latest generation of Databricks Knowledge Assistant.

We’ll share how rethinking retrieval as a trainable, inference-time system lets Knowledge Assistant search more broadly, rank more intelligently, and serve answers faster. The talk covers the major ingredients behind this shift: agentic data generation, retrieval-specialized training, realistic quality validation, and serving optimizations designed for production workloads.

Together, these advances push the quality–cost–latency frontier for enterprise search, enabling Knowledge Assistant to deliver faster, higher-quality retrieval for both direct answers and more complex agentic workflows.

Abstract:

Modern foundation models are no longer defined by scale alone. Training stacks now need to support Mixture-of-Experts (MoE) architectures, long-context workloads, multimodal data, and modern capabilities on rapidly evolving GPU platforms, all while staying fast, reliable, and portable across clouds. This talk goes inside the Databricks training stack and the systems and research work behind it.

To maximize efficiency on the hardware, we integrate high-performance kernels and efficient optimizers; enable low-precision training and runtime optimizations for both long-context and MoE training. To scale training to thousands of GPUs, we built a composable 4D parallelism stack. This enables large MoE models through expert parallelism, ultra-long context training through hybrid sequence parallelism, and scale-out training with data and pipeline parallelism. Our stack runs seamlessly on both Hopper and Blackwell GPU platforms across major cloud vendors.

With these optimizations, the engine supports training frontier model families such as GPT-OSS, Qwen and Kimi, and powers Databricks products through custom-trained models.

Abstract:

AI agents need more than prompts, tools, and retrieval. In production, they also need transactional state, predictable latency, and a development workflow that makes iteration safe. This talk looks at Lakebase as a Postgres-native system for building agentic applications on Databricks. I’ll cover the architectural choices behind Lakebase, including compute-storage separation, zero-copy branching, fast start-up, and independent scaling for read and write workloads. I’ll also show why those choices matter for developers and coding agents: isolated environments for experimentation, low-latency operational state, and a cleaner path from application data to analytics and AI without stitching together a separate OLTP stack and ETL pipeline. The goal is to give developers and researchers a concrete mental model for the data layer beneath modern agents, and why a transactional substrate changes what you can reliably build.

Abstract:

Engineering is shifting from human-driven to agent-driven, and the teams winning right now are deploying coding agents as aggressively as they can. The catch: a coding agent is only useful when it can read your tickets, your design docs, your customer issues. And the second you hand it that access, the hard problems start. An MCP-connected agent quietly becomes the most privileged developer in your org, with no governance. Agent spend climbs into top R&D line item territory, split across a half-dozen tools with no shared budget or rate limits. And while everyone's racing to adopt AI, everyone's using something different, so no executive can honestly answer who's using it or whether it's working, So how do you actually move fast on this without giving up on governance? In this talk we'll walk through how we're solving it at Databricks with Unity AI Gateway.

Abstract:

Training and fine-tuning AI models shouldn’t require becoming a GPU infrastructure expert. As enterprises scale AI beyond prototypes, managing clusters and distributed compute has become a major bottleneck.In this session, we’ll introduce AI Runtime, a new serverless GPU training experience on Databricks that provides on-demand NVIDIA A10 and H100 GPUs directly on Lakehouse data. You’ll see how teams can train and fine-tune LLMs, recommendation systems, forecasting models, and computer vision workloads without managing infrastructure.We’ll cover how AI Runtime combines optimized GPU compute and distributed training with native support for frameworks like PyTorch and Hugging Face, alongside MLflow, Unity Catalog, and Lakeflow to unify the full AI lifecycle.You’ll learn how to reduce infrastructure overhead, accelerate iteration cycles, and standardize end-to-end AI workflows from experimentation to production.

Abstract:

Every successful developer platform earns its place by getting a handful of primitives right. Agents are the next layer of the stack — and we've built the platform underneath them. This talk walks through the five public APIs that make up our agentic infrastructure: an inference gateway, a memory API, an agent control plane, agent deployment, and sandbox management. Each layer stands on its own, composes cleanly with the others, and is built to get out of the developer's way. For each component, we'll cover the design decisions, the tradeoffs we navigated, and the lessons we borrowed from other platforms. We'll close by walking through how our own internal agents (the ones we ship inside Databricks every day) are built on exactly this stack.

Abstract:

Coming Soon!

Abstract:

Agent development has a testing problem: outputs are non-deterministic, runs take minutes, and ""correct"" is a judgment call.  At Databricks, we built coSTAR (coupled Scenario, Trace, Assess, Refine) on MLflow to treat agent development like software development — scenarios as test fixtures, MLflow traces as test output, LLM judges as the test suite, and a coding assistant running the red-green loop.

The key insight: coSTAR runs two coupled loops. One refines the agent against judges. The other aligns the judges against human expert assessments to ensure that tests can be trusted. coSTAR is built on MLflow, works with any LLM or agent framework, and enabled us to go from two-week manual reviews to hours when launching agents.

Abstract:

Your AI agent is in production. How do you actually know it's working? Most teams answer with vibes or sporadic human review, neither of which scales. LLM judges close that gap, but a judge is only as good as the definition of "quality" it encodes, and that definition lives in your Subject Matter Experts’ (SME) heads. Over the last 12 months we have run a hands-on workshop with several customers across financial services, public sector, healthcare-adjacent, retail, voice AI, manufacturing, and utilities, helping them turn tacit SME judgment into aligned LLM judges that ship to production. This talk walks through a few customer stories, the process, and the lessons learned.

Abstract:

What if your AI coding agent could build its own report card? We show how we used Genie Code to bootstrap the evaluation pipeline of Genie Code itself — setting up MLflow trace analysis and trace annotation end-to-end. It's turtles all the way down, and it actually works.

Abstract:

As foundation models become more capable, the bottleneck for AI agents has shifted from reasoning to grounding: giving the agent the right context for the task. Yet most agents remain stateless: they forget preferences, repeat mistakes, and solve the same problems from scratch. This talk introduces memory scaling, demonstrating how agent performance improves as it accumulates and reuses past experience. I’ll share how Databricks is approaching this—using memory as a plugin, via agentic tools, and through offline pipelines—with proven results in building LLM judges, text-to-SQL, and long-horizon QA. Finally, I’ll address the practical challenges of making memory reliable, scoped, and secure in production.

Abstract:

Today’s frontier AI systems can ace Math Olympiad problems and answer questions requiring highly-specific domain knowledge, but can they reliably answer questions over the complex, unstructured data enterprises actually have? 

In this talk, we’ll introduce OfficeQA, a benchmark for evaluating end-to-end grounded reasoning over the large, heterogeneous document corpora like those found in enterprise settings. Built from the U.S Treasury Bulletin corpus, a complex dataset spanning 89,000 pages with almost 100 years of financial accounts from 1939 through today, OfficeQA evaluates whether AI agents can retrieve evidence across large document collections, parse scanned and digital PDFs, interpret complex tables and produce precise numerical answers. Rather than highlighting capabilities over trivia domain knowledge or PhD puzzles, this benchmark reflects the realistic, economically valuable tasks that resemble the questions analysts ask about their data.

The results show that agentic enterprise reasoning is still far from solved. We’ll discuss where systems break down (parsing errors, retrieval failures, visual reasoning gaps, and precision mistakes) and where future research is still needed. We also highlight why OfficeQA matters to the industry, growing as a featured benchmark across standardized evaluations by OpenAI, Anthropic and Gemini under professional knowledge work tasks. We’ll end by previewing the Databricks Grounded Reasoning Cup, where student teams alongside frontier AI labs will compete live at DAIS to push the state of the art in grounded reasoning using a new corpus released in partnership with the U.S Treasury for the 250th anniversary of the United States.

Abstract:

Data agents in the enterprise must navigate semantic context across hundreds of thousands of tables, notebooks, dashboards, and documents, deeply understand relevant assets and their relationships, surface ambiguities and assumptions, run an analysis, and report back---ideally within a few minutes. This talk examines how the challenges facing data agents differ from those of traditional coding agents, and some approaches Databricks is taking to address them. I'll focus on enterprise context and asset discovery, agent architecture (including multi-LLM systems), and the optimization and evaluation of data agents.

Abstract:

Standard tool-calling agents face a structural limit on multi-step reasoning. Every tool output is serialized to JSON, appended to the conversation, and re-read on each turn. As hops accumulate, the context window fills with verbose intermediate state and task performance degrades.

MemEx replaces the JSON loop with a persistent Python kernel. Tools are auto-injected as Python functions with their schemas preserved, and outputs remain as typed objects in a scope that carries across turns, so large documents or million-row dataframes can be held in memory and manipulated programmatically. Only what the agent passes to `print()` materializes as tokens. The same primitive enables tool composition through variables and sub-agents that share the parent scope.

Across nine frontier and open-weight models on two enterprise agentic tasks (OfficeQA Pro and Enterprise Structured Retrieval), frontier models (Opus 4.6, Sonnet 4.6, Gemini 3.1 Pro) gain 2 to 5 accuracy points at 25 to 30% lower cost, and Qwen 122B and Qwen 397B nearly double accuracy at 40 to 50% lower cost. Four of the five points on the OfficeQA Pro cost-accuracy Pareto frontier are MemEx configurations.

MemEx is rolling out across Agent Bricks and first-party Databricks agents.