agentic ai

• youtube - @googlecloudtech 

• youtube - @GoogleDevelopers

•  youtube - @googledeepmind 

• Google Cloud - Topics  

Pillars

• Operational Excellence 

• Security, Privacy & Compliance 

• Reliability 

• Cost Optimization 

• Performance Optimization 

• Sustainability 

• Cross Pillar Perspectives

  • Technology - AI/ML

  • Domain - FSI, Retail

  • Sector 

Components

• Principles - Define major goals of your cloud architecture  

• Practices - Understabd trade-offs

• Questions & Recommendations - Effective improvement plans

• Interactive Workshops 

Anatomy of Agentic System

• The anatomy of an agentic system outlines the core structural components and operational modules required to transform a static model (like a Large Language Model) into an active, decision-making agent capable of executing multi-step workflows.

1. The Core Decision Engine ("Brain")

The backbone of the system—typically a Large Language Model or specialized multimodal foundation model. It processes inputs, maintains system prompts/instructions, evaluates contexts, and decides what actions to take next.

2. Planning & Reasoning Module

Responsible for breaking down complex human intents into manageable, logical execution steps:

• Task Decomposition: Splitting a high-level goal into sequentially ordered sub-tasks or sub-goals.

• Self-Reflection & Self-Correction: Evaluating intermediate results against target criteria (e.g., verifying if generated code compiles, or if retrieved data answers a sub-query) and revising the execution strategy when errors occur.

• ReAct Framework: Iterating through a loop of Reasoning (thinking about what to do next) $\rightarrow$ Acting (invoking a function) $\rightarrow$ Observing (analyzing the environment's feedback).

3. Memory Subsystem

Allows the agent to retain context and learn across operations:

• Working / Short-Term Memory: The active context window, scratchpad, or prompt thread tracking current operational state, intermediate steps, and immediate observations.

• Episodic & Long-Term Memory: External databases (often Vector DBs with RAG capabilities or Knowledge Graphs) that persist past experiences, user preferences, domain knowledge, and execution histories across sessions.

4. Tool Integration & Action Execution (Capabilities)

The interfaces through which the agent interacts with external digital systems:

• API Integrations & Function Calling: Reusable interfaces to query databases, call REST APIs, send emails, or manipulate files.

• Environment Executors: Sandboxed environments (e.g., Python code interpreters, terminal shells, web browser automations) where the agent runs commands, inspects outputs, and catches execution exceptions.

5. Guardrails & Governance Layer

The safety mechanism surrounding the core agent loops:

• Autonomy Dials: Threshold controls determining whether an agent can execute an action autonomously or requires Human-in-the-Loop (HITL) approval (e.g., auto-executing read operations vs. requiring authorization for financial transactions).

• Policy & Validation Filters: Input/output guardrails preventing prompt injection attacks, sensitive data exfiltration, hallucinated tool invocations, or policy violations.

6. Multi-Agent Orchestration (Optional Extension)

In enterprise architectures, a single agent is often part of a broader agentic network:

• Conductor / Orchestrator Agent: Accepts user requests, decomposes them, and delegates tasks to domain-specific worker agents.

• Specialized Worker Agents: Focused agents specialized in single domains (e.g., Data Retrieval Agent, Code Reviewer Agent, Policy Validator Agent).

What is an Agent ?

• • A self contained unit that can reason, act and achieve a goal autonomously 

  • AI Mode + Instructions + Tools 

  • Higher Complexity leads to workflows 

    • Agent A -> Function -> Agent B

  • The Agent decides which tool to call and when ?

  • Unlike standard foundation models that simply process inputs and generate text responses, an agent combines reasoning capabilities with execution mechanisms to handle non-linear, multi-step workflows.

Key Characteristics

• Goal-Oriented Autonomy: Given a high-level objective, the agent evaluates what steps are required, determines the correct sequence of actions, and executes them without needing manual guidance at every step.

• Dynamic Decision-Making: Instead of following a hardcoded script, the agent dynamically decides which tools or functions to call, when to invoke them, and how to process the feedback returned by the environment.

• Workflow Delegation: As complexity increases, individual agents can be connected into multi-agent workflows—passing tasks, parameters, or intermediate outputs from one specialized agent to another ($\text{Agent A} \rightarrow \text{Function} \rightarrow \text{Agent B}$).

Agent Design Patterns 

• ReAct (Reason+Act+Observe) - Think -> Act-> Observe -> Repeat 

  • The foundational execution pattern where the agent operates in an iterative loop:

  • Think (Reason): The agent analyzes the current state, consults its instructions, and determines what needs to be done next.

  • Act: It invokes a tool, API, or function based on that reasoning.

  • Observe: It captures the output or error returned by the environment.

  • Repeat: It feeds that observation back into its context window to decide the next step until the goal is achieved.

• Human-in-the-Loop  (HITL)

  • Agent -> Human Review -> Approval -> Execution.

  • For high-risk operations (such as executing a financial transaction, modifying production cloud infrastructure, or sending external emails), the agent prepares the payload but pauses execution until an authorized user reviews and approves the action.

• Reflection and Self-Correction

  • A pattern that introduces quality control loops into the agent's workflow:

• The agent generates an initial draft, code snippet, or plan.

• Instead of immediately returning it, a secondary prompt or inner loop has the agent critique its own output against constraints, tests, or rubrics.

• If flaws or syntax errors are found, the agent rewrites or refines the output before presenting it downstream.

• Tool Use / Function Calling

• The pattern that equips an LLM with external capabilities:

• The model is provided with a schema defining available APIs, calculators, code interpreters, or databases.

• Rather than hallucinating an answer, the model recognizes when it lacks real-time data or computational precision, pauses generation, formats a structured payload for the tool, and uses the resulting payload to construct the final response.

Planning & Task Decomposition

• Used for complex, multi-layered objectives:

• Given a high-level prompt, a planner agent breaks the overarching goal into an ordered sequence of sub-tasks (e.g., Step 1: Query database $\rightarrow$ Step 2: Format results $\rightarrow$ Step 3: Generate summary report).

• The agent executes the plan step-by-step, dynamically adjusting subsequent steps if an earlier task yields unexpected results.

Agent Challenges 

Deploying and scaling agentic systems introduces complex engineering and governance hurdles that go far beyond traditional software engineering or single-turn generative AI.

1. Predictability & Behavioral Drift

• Non-Linear Execution Paths: Because agents dynamically choose tools, re-plan steps, and adapt to intermediate feedback, two identical user inputs can trigger completely different execution paths.

• Hallucinated Tool Invocations: An agent might misinterpret an API documentation schema, pass incorrect parameters to a backend function, or select an entirely wrong tool to solve a sub-task.

• Compounding Errors: If an agent makes a minor reasoning error early in a 10-step autonomous loop, that error propagates and magnifies, leading to compounding failures or silent crashes down the line.

2. Stability & Infrastructure Resilience

• Distributed State Management: Maintaining a consistent state, working memory, and execution history across distributed microservices becomes critical. If a node fails mid-task, recovering context without restarting the entire workflow is notoriously difficult.

• Infinite Loops & Cost Explosions: Agents caught in self-reflection loops can get stuck endlessly retrying failed tool calls, rapidly driving up token consumption, latency, and cloud infrastructure expenses.

• Thundering Herd & Concurrency: Multi-agent architectures executing machine-speed requests can easily overwhelm downstream legacy databases and third-party APIs.

3. Operations & Observability

• Tracing "Thinking" vs. Code: Traditional application monitoring tracks code execution lines and explicit logs. Agentic monitoring must capture non-deterministic reasoning traces, decision logic, and agent-to-agent communication dynamics.

• Complex Testing: Standard unit tests are insufficient because they catch static code bugs rather than emergent behavioral failures that only appear under real-world production stress.

4. Security, Governance & Blast Radius

• Unintended Autonomous Actions: If an agent has write or delete permissions, a misinterpretation can result in unauthorized data modifications, unintended financial transactions, or corrupted system records.

• Prompt Injection & Jailbreaks: Attackers can inject malicious instructions via external data sources (such as an email an agent reads or a website it browses), hijacking the agent's control flow.

• Granular Identity Management: Tracking which agent, acting on whose behalf, invoked a specific privileged API requires robust, audit-grade identity chains across distributed systems.

RAG (Retrieval-Augmented Generation)

RAG (Retrieval-Augmented Generation) is an architecture that equips Large Language Models with external, real-time data retrieval capabilities to ground generated responses in verified information.

Instead of relying solely on the model's static pre-trained weights, RAG dynamically fetches relevant documents or data chunks from an external knowledge source before generating a response.

Feed the LLM relevant context in real-time, by using as information retrieval system. 

Core Mechanics

Chunking & Embedding: External knowledge bases (documents, databases, manuals) are split into smaller segments (chunks) and converted into vector embeddings.

Retrieval: When a user submits a query, the system embeds the query and searches the vector database for the most semantically relevant context chunks (using similarity metrics like cosine distance).

Augmentation: The retrieved context chunks are inserted directly into the prompt alongside the original query.

Generation: The LLM processes the augmented prompt, synthesizing an accurate response strictly derived from the retrieved facts.

Key Benefits

• Eliminates Hallucinations: Prevents the model from fabricating facts by forcing it to answer using provided source material.

• Real-Time Data Access: Bypasses the need for costly model retraining or fine-tuning when enterprise or external data changes.

• Auditability & Attribution: Allows systems to cite specific source documents and records, enabling clear traceability and compliance checks.

• Access Control: Ensures data permissions are enforced at the retrieval layer, restricting access to sensitive documents based on user authorization.

Embeddings 

• Embeddings are numerical representations of data—such as words, sentences, images, or audio clips—encoded as dense vectors of floating-point numbers in a high-dimensional space.

• Instead of treating text as discrete keywords or strings, an embedding model maps semantic meaning into coordinates. Concepts, words, or documents with similar meanings or contexts are positioned close to one another in this vector space, while unrelated concepts are placed far apart.

Open Knowledge Format (OKF)

• A Bundle is a directory of concepts 

• One file is one concept 

• Concept type is free form - Notaxonomy for the Spec 

• Every file has 2 halves 

  • Formatter

    • type, title, descirption, resource, tags

    • generatedm verified, status, stale_after, sources

  • Body 

ADK - Agent Development Kit (Google AI Framework)

• Specilized for building, executing and managing agents 

• Native understanding for AI agents

• Pre-itnegrated with primitives: Tools, State, Workflow & Skills 

What is the Skils ?

• Progressive Disclosure 

• Policy skill, Validation skills, Summary skill 

Core Architecture

• Tools - Actions

• State - memory

• Workflows - coordination 

• Skills - Reusability 

Why Skills ?

• • Efficient scaling

  • Contextual logix 

  • Reusable Expertise 

  • Progressive Disclosure

    • Metadta - Name, descriotion, discovery info

    • Instructions - Full skill bidym operating rules and task guidance 

    • Resources - References, Assettsm and scrips. 

Common Skill Patterns 

• Inline - directly in code 

• File based - folders with SKILL.md + resources 

• External - Repositories / Packages

• Meta - Skill that can create/manage other skills 

Agent CLI

• Idea -> Project setup-> agent skeleton -> configure -> run & test -> iterate 

Tool Calling 

• Tool is a funtion the model is alloed to call e.g. API

• Calls could be Sequential or Parallel

• LLM reads the user request , picks the tool, fills om the arguments, make the call and read the results back

• 3 kinds of Tools

  • Retrieval / Read (Read only)

  • Action / Write (Change / Update)

  • Computation (Deterministic)

Tool Schema 

• function name = tool name

• docstring =  description the model reads to know when to use it 

• type hints = parameter types

• return type/dict = shapes what model can expect back

Binding 

• Binding = passing your verified functions into the agent's tools list 

• Unbound functions are invisible to agent 

Prompt does the steering 

• map user intent = which tool to call 

MCP (Model Context Protocol)

modelcontextprotocol.io - intro 

Model Context Protocol (MCP) is an open, standardized communication protocol designed to securely connect AI models and agents to external data sources, enterprise tools, and execution environments.

Instead of writing custom point-to-point integrations for every AI model and database pair (M X N integration problem), MCP acts as a universal API bridge—similar to how USB-C standardizes hardware connections.

• Connecting Agents to the world's informaiton & services

• USB-C for AI - Open protocol standardizing applications context / Tools for LLMs

• Secure and Controlled Data Acess 

• Client / Server Arhchtecture 

Core Architecture

MCP operates on a client-server model over JSON-RPC 2.0:

• MCP Host: The runtime environment or client application initiating the AI session (e.g., IDEs like VS Code/Cursor, desktop AI assistants, or agent orchestration frameworks).

• MCP Client: A protocol instance running inside the host that maintains a dedicated 1:1 connection with an MCP server.

• MCP Server: A lightweight server wrapper exposing capabilities from external systems (e.g., Postgres databases, GitHub repos, local filesystems, or Google Cloud services).

Key Operational Benefits

1. Decoupled Architecture: Developers build an MCP server once for a service (e.g., a Jira or BigQuery MCP server), and any MCP-compliant AI client can immediately use it without custom glue code.

2. Context Granularity & Security: The protocol isolates execution boundaries. Data access occurs strictly within the security context and permissions configured for the local/remote MCP server.

3. Dynamic Discovery: When an AI session starts, the MCP client queries connected servers to dynamically discover available tools, resources, and prompt templates.

What an MCP Server exposes 

• tools/list 

• prompts/list 

• resources/list 

A2A - Agent to Agent Protocol

A2A (Agent2Agent Protocol) is an open, framework-agnostic communication protocol designed to allow autonomous AI agents built on different platforms, vendor stacks, or frameworks to discover each other, securely exchange context, and collaborate on complex multi-step tasks. 

Core Architecture & Mechanics

A2A uses a client-server paradigm built on web standards like JSON-RPC 2.0 over HTTPS and Server-Sent Events (SSE) for real-time task streaming:

• Agent Cards (/.well-known/agent.json): A JSON discovery manifest exposed by every remote agent. It acts like an agent's public business card—advertising its capabilities, supported modalities, input schemas, and security requirements.

• Client Agent (A2A Client): The delegating agent or orchestrator that initiates communication by reading a remote agent's Agent Card and submitting work requests.

• Remote Agent (A2A Server): The executing agent that accepts tasks, runs domain logic internally, and updates task states.

• Tasks & Artifacts: Work is tracked as structured Tasks with explicit lifecycle states (e.g., submitted, in-progress, input-required, completed, failed). Outputs (such as documents, code, or datasets) are wrapped in typed Artifacts.

What is a multi-agent system ?

• A Multi-Agent System (MAS) is an architectural framework where multiple autonomous AI agents—each possessing distinct capabilities, roles, tools, or domain specializations—collaborate, communicate, and coordinate to solve complex problems that exceed the capacity of a single agent.

• Rather than relying on a single large language model to handle an entire end-to-end workflow (which risks context window bloat, halluciations, and compounding errors), a multi-agent system divides labor across smaller, focused units.

• A system os specialized AI agents to solve complex taks, rather than relying ona single model to do everything 

Modular Architecture

• • Components scale, test and deploy indepedently 

  • Making it easier to scale, test and reuse code 

Multi-Agent Design Patterns 

Multi-agent patterns structure how autonomous agents divide responsibilities, route information, share memory, and collaborate to achieve complex objectives.

• • Sequential 

    • Best for: Fixed order, repeatable 

  • Parallel - tasks are executed at the same time 

    • Best for: Indepedent, Concurrent 

  • Loop - if the exit condition is not true, then the loop continues 

  • Review & Critique - Response is evaluated and quality score sent 

    • Best for: Needs a validation step

  • Iterative Refinement - if quality score doesn't meet requirements, prompt is enhanced

    • Best for: Improve over cycles 

Dynamic Patterns 

• Co-ordinator - dynamically routes request to the appropriate subagent 

  • Best for: Adaptive routing, varied input 

• Hierarchical Decomposition - from co-ordinator agent 

  • Best for: Ambiguous, multi-level planning 

• Swarm - if exit condition is not met, subagents in the swarm continue to collaborate on the task 

  • Best for: Debate towards synthesis 

Core Structural Topologies

1. Orchestrator-Worker (Hierarchical)

A central supervisor or router agent accepts the overarching goal, decomposes it into specific sub-tasks, assigns them to specialized worker agents, evaluates their outputs, and synthesizes the final response.

• Best for: Complex tasks requiring centralized planning, strict goal alignment, and heterogeneous domain expertise.

• Example: An enterprise report generator where the Orchestrator assigns SQL retrieval to Agent A, statistical modeling to Agent B, and PDF formatting to Agent C.

2. Sequential Pipeline (Assembly Line)

Agents pass work downstream in a fixed, linear order. The output of Agent A becomes the context/input for Agent B, which then feeds Agent C.

• Best for: Standardized workflows with clear, deterministic stage gates.

• Example: Content publishing pipelines (Topic Research -> Article Drafting -> Style & Fact Editing -> SEO Optimization).

3. Peer-to-Peer / Collaborative (Debate & Review)

Agents with distinct roles or personas interact symmetrically, reviewing, debating, or refining each other's work until a consensus or quality threshold is met.

• Best for: High-stakes tasks requiring rigor, error correction, and iterative refinement.

• Example: Autonomous software engineering where a Coder Agent writes code, a Reviewer Agent runs test cases and flags bugs, and a Security Agent scans for vulnerabilities before approval.

4. Parallel Mesh (Fan-Out / Fan-In)

The system fans out independent sub-tasks simultaneously across multiple identical or specialized agents to process data concurrently, then fans in to aggregate the results.

• Best for: Latency-sensitive multi-source data collection, competitive market scanning, or parallel web research.

Key Components of a Multi-Agent System

• Role Definition & System Prompts: Each agent is scoped with a clear identity, domain instructions, constraints, and operating guidelines.

• Shared Memory & State Synchronization: Systems maintain a unified state manager or persistent message bus to share context, execution history, and artifact outputs across agent boundaries.

• Inter-Agent Communication Protocol: Standardized message formats and execution interfaces (such as function calls or JSON payloads) that allow agents to trigger actions or pass data back and forth.

• Hand-off & Control Flow Mechanics: Explicit logic governing when an agent completes its task, hands off control to another agent, or escalates back to a supervisor or human operator.

Evaluation (Test)

Evaluating AI agents requires testing non-deterministic, multi-step execution paths rather than simple single-turn text outputs. Because agents act autonomously using planning, memory, and tool execution, an agent can reach the correct answer through a dangerous sequence of actions—or fail entirely due to a minor error compounding early in its trajectory. 

Agentic Development Life Cycle (ADLC)

• Design

• Develop 

• Evaluation (Test)

• Deploy

Evaluation Landscape 

• Output Quality 

  • • Vibe Check 

    • Rubric Scoring

    • Golden Set Comparison

• Operational Metrics 

  • • Bias Detection

    • Toxicity Scan

    • Adversarial Testing

    • Latency Analysis 

    • Throughput Testing

    • Resource Utilization 

• Cost & Efficiency 

  • • Token Count

    • Model Size Optimization 

How we determine success in evaluation 

• Identify and map KPIs

  • Alignment with business outcomes 

• Establish metrics and create rubrics 

  • Strong Operational Definitions 

  • Sensitivity 

  • Specificity 

• Execute the evalution 

  • Choose a technique 

    • Deterministic Metrics (Hard Gates)

    • Human Based 

    • Model-Based Metrics (Auto-raters)

Building Eval Pipeline 

1. Collect Golden Dataset

   • Curate a golden dataset of 100–500 realistic edge cases, complex multi-step user intents, and past production failure traces. Avoid synthetic single-turn prompts.

2. Isolated Component Unit 

   • Test low-level primitives independently before evaluating agent loops:

• Retriever (RAG): Measure Context Relevance and Faithfulness.

• Tool Selection: Verify parameter syntax, JSON schema validity, and tool routing accuracy.

3. Evaluate Full Trajectories and Evals 

   • Execute agents in sandboxed environments and evaluate both the final Outcome (Goal Completion) and the Trajectory (Reasoning, tool sequence efficiency, safety policy adherence).

4. CD/CD Merge Quality Gates

   • Hook evaluation suites into pull request builds. Calibrate automated LLM judges against human annotations to maintain a target agreement rate (e.g., >85%). Block builds that regress on key safety or accuracy baselines.

5. Product Sampling and Drift 

   • Sample 5–10% of live traffic traces in production. Automatically surface failures, out-of-scope queries, or loop retries to continuously feed back into the golden dataset.

Anatomy of an Eval Case

• Eval ID = unique name 

• Session input = user and session the eval starts from

• User content = the message 

• Final response = the golden expected answer

• Tools uses = expected tools and arguments called

Key Challenges & Core Issues

Testing autonomous agents introduces unique operational hurdles compared to conventional software:

1. The Compounding Error Cascade

A minor planning error or slightly inaccurate parameter on Step 1 propagates and magnifies over an 8-step run. Inspecting only the final output masks the root cause of the failure.

2. Trajectory Non-Determinism

Because agents continuously evaluate tool feedback, two identical runs can follow completely different execution paths to reach the same correct answer. Single-run unit tests are insufficient; evaluation requires calculating variance over repeated passes (Pass@K).

3. LLM-as-a-Judge Biases

Automated evaluator models exhibit systemic biases:

• Position Bias: Favoring candidates presented first/last in A/B testing.

• Verbosity Bias: Assigning higher scores to longer, wordier reasoning chains regardless of correctness.

• Self-Preference: Evaluators tend to score outputs generated by their own family of models higher.

4. Failure Attribution in Multi-Agent Topologies

In multi-agent systems (e.g., Orchestrator $\rightarrow$ Sub-Agents), assigning blame for an error is complex. Did the Orchestrator route to the wrong specialist, did the specialist fail tool invocation, or was handoff state truncated?

ADK Eval 

ADKs built-in framework for testing agents automatically 

• Capture 

  • Turn real conversation into a golden dataset

    • Golden Dataset 

      • Quuery 

      • Trajectory 

      • Response 

• Score

  • Grade trajectory + response against thresholds 

    • Trajectory - Tool Path - Did it call the right tools in the right order ?

    • Response - Final Outut - Is the answer correct & grounded ?

• Automate 

  • Wire it into CI/CD so regressions get caught 

What is Hill Climbing ?

• Gather data

• Measure Metrics 

• Iterative Improvement 

Hill Climbing Approaches 

• Changing prompts (Prompt Optimization)

• Changing Model Behavior 

• For Complex systems, break down individual components and measure overall performance 

• Rearchitecting flows/prompt chains / agents 

Agent - Skills 

An Agent Skill is a modular, reusable package of procedural instructions, domain context, templates, and execution scripts that extends an AI agent's capabilities.

While the Agent acts as the runtime decision maker (the engine), a Skill represents the packaged expertise and procedural memory (the playbook) that tells the agent how to execute a specific task accurately.

Core Distinction: Agent vs. Tool vs. Skill

• Agent: The autonomous entity that reasons, plans, and manages execution loops.

• Tool: An API connection, database query driver, or shell runner that allows the system to take an action.

• Skill: The structured knowledge layer that teaches the agent when, why, and in what sequence to invoke those tools according to domain standards.

Key Architectural Characteristics

1. Progressive Disclosure: Rather than bloating the system prompt with thousands of lines of instructions on every turn, skills are loaded dynamically only when triggered by relevant tasks, saving context window capacity and reducing token overhead.

2. Deterministic Execution: Skills often bundle explicit validation logic, local scripts (Python/Bash), or exit criteria to enforce consistency and prevent hallucinations in complex tasks.

3. Open Specification & Portability: Standardized as directory structures containing human-readable files (like Markdown instructions and local assets), skills can be version-controlled, audited, and shared across agent harnesses.

Anatomical Structure of a Skill Directory

A standard agent skill is typically packaged as a lightweight filesystem directory:

• SKILL.md: Defines the activation criteria, workflow stages, safety checks, and step-by-step logic the model follows.

• references/: Documents loaded into memory only when specific sub-tasks require deeper contextual background.

• scripts/: Offloads computation, deterministic math, or precise validation from the probabilistic LLM to local scripts.

Skills Patterns

• Informational - provides additional context

• Tool Wrapper - provides instructions on jow to use existing tool

• Generator - guides agents on producing content of a specific type or format 

• Reviewer - defines criteria for asessing the work of himands or agents 

• Workflow - orchestrates a repeatable sequence of tasks 

Context Advantages: Progressice Disclosure 

• When - YAML formatter 

• How - Skill.md body 

• What - References / Assets files 

Context Engineering 

Context Engineering is the practice of systematically designing, structuring, retrieving, and managing the information fed into a Large Language Model’s context window so that it can reason, execute tools, and make accurate decisions.

While early AI development focused heavily on Prompt Engineering (crafting the perfect wording, persona, or phrasing of a single prompt), Context Engineering shifts the focus to the entire state and payload surrounding the model. It addresses what information enters the context window, when it is loaded, how it is formatted, and how it is updated across multi-turn autonomous execution loops.

The dynamic assembly of information ro make agents stateful. 

Key Pillars of Context Engineering

1. Information Retrieval & Filtering (Dynamic RAG): Determining which chunks of long-term memory, vector embeddings, or database records are relevant to the immediate sub-task, ensuring the model isn't overwhelmed by irrelevant background details.

2. Progressive Disclosure: Loading high-level metadata (such as skill catalogs or tool descriptions) upfront, and fetching full procedural instructions (e.g., standard operating procedures, schemas) only when triggered by specific user intents.

3. State & Scratchpad Management: Maintaining an active working memory within multi-agent or multi-turn execution loops so the model can track what it has already accomplished, what step comes next, and what errors it encountered.

4. Context Compression & Pruning: Truncating, summarizing, or stripping old observation logs, chat histories, or chatty inter-agent messages to prevent hitting context limits, reducing latency, and avoiding "lost-in-the-middle" attention degradation.

5. Schema & Protocol Standardization: Structuring information using standardized interfaces—such as the Model Context Protocol (MCP)—so external databases, files, and tools flow into the context window securely and consistently.

3 levels of Context 

• Persistent (System Instructions) - Foundational Rules 

• Semi-persistent (memory) - history of the conversation 

• Transient (Dynamic Data) - Turth injected from the outside world in real-time 

Four Principles of Context Engineering 

• Write - put context outside the window

• Select - Pull only what's needed in 

• Compress - Keep only required tokens

• Isolate - Split context across boundaries. 

Spec Driven Development 

• Product - prd.md / spec.md 

• UX/Design - design.md

• Project Rules - GEMINI.md

• Planning - plan.md

• Capability - skills/*.md - SKILLS.md 

• SRE / Day 2 - runbook.md 

• Project - Readme.md

• The Repo - AGENTS.md  

Harness Engineering 

Harness Engineering is the practice of designing, building, and maintaining the software environment (the "harness") that surrounds, controls, and evaluates an AI agent or language model.

While Context Engineering focuses on curating the prompts and data payload fed into the model's context window, Harness Engineering focuses on the surrounding software architecture—managing execution loops, sandboxing, session state persistence, safety guardrails, tool dispatching, and error recovery.

Key Responsibilities of a Harness Architect

1. Runtime Execution & State Management: Operating the non-deterministic reasoning loop (e.g., ReAct or state machine loops), keeping track of active tool executions, handling pauses for human intervention, and persisting execution state if a system crashes.

2. Environment Sandboxing & Tool Dispatching: Providing isolated execution environments (such as Docker containers or WASM runtimes) where an agent can run generated code, execute shell scripts, or call APIs without risking system security or cloud infrastructure.

3. Guardrails & Safety Control (Autonomy Dials): Enforcing policy limits before tool calls reach production systems. The harness intercepts actions to verify parameters, enforce read-only boundaries, or escalate high-risk operations for Human-in-the-Loop (HITL) approval.

4. Error Interception & Recovery: Catching API timeouts, rate limits, schema validation errors, or malformed JSON tool calls from the model, framing those failure logs cleanly, and feeding them back into the model for self-correction without crashing the session.

5. Evaluation & Observability Integration: Hooking into telemetry and tracing frameworks (e.g., OpenTelemetry, LangSmith, or Phoenix) to log trajectory steps, token costs, latency, and tool accuracy for offline benchmark testing.

Agent Deployment 

• Agent Runtime

  • Managed sessions + memory, ADK Tool Auth, Obervability 

  • Best for: Durable state (Long-lived / memory depedent conversation) no infra work

  • State: Auto-creates a managed session (VertexAiSessionService)

  • Cold start: sub-second 

  • Billing: Per vCPU-hour +GB-hour

• Cloud Run 

  • Serverless container, real endpoint (Https), Scales to zero

  • Best for: Custom container, sidecars, predictable cost (scale to zero) 

  • State : In-process memory by default - a restart wipes the conversation. You have add a session services with your own Firestore / Postgres backend to build and operate.

  • Cold-start:  cold start on wake 

  • Billing:  Pay-per-use 

• GKE 

  • Kubernetes, Full control (and run) everything, Custom Networking, Open Models

  • Best for: Enterprise scale, special networking, open models 

• Gemini Enterprise (GE)

  • No code, Just UI with no custom tools    

Core Governance Components

The following Agent Platform components work together to provide a unified governance architecture:

• Agent Identity (Who made the request?): Assigns a unique, secure ID (a SPIFFE ID) to each agent. This identity acts as the agent's digital signature for authentication, access control, and auditing.
  Agent identities are secured by default with Context-Aware Access which enforces end-to-end cryptographic authentication by using mTLS and DPoP.

• Agent Registry (What destinations are approved?): Acts as the central directory for all approved agents, tools, Model Context Protocol (MCP) servers, and endpoints (such as essential Google Cloud APIs) in your organization. Agent Gateway uses this directory to check permissions before allowing connections.

• Policies (What actions are permitted?): Lets you implement rich sets of agentic security and governance policies that control which agents can reach specific resources and what content is allowed to pass through:

  • IAM Unified Access Policies: Access rules that link an agent's identity to approved tools and endpoints in Agent Registry. By default, all connections are blocked unless an explicit IAM policy grants access.

  • Model Armor: Content security filters attached to gateways. Model Armor scans user prompts and tool responses in real time to block prompt injection attacks, sensitive data leaks, and harmful content.

  • Semantic Governance Policies: Rules written in plain language that control how agents use tools. They enforce business rules at runtime to prevent agents from taking unintended actions, such as running unsafe combinations of tools.

  • Custom authorization engines: Custom authorization extensions that let you delegate authorization decisions to third-party systems by using Service Extensions.

• Internally, all of these policy types are enforced by using authorization policies managed through Service Extensions.

• Agent Gateway (Where are policies enforced?): Serves as the main entry and exit point for network traffic between clients-to-agents, and agents-to-anywhere. It manages encrypted connections (mTLS), translates protocols (such as MCP, REST, and gRPC), and applies policy checks to all traffic. To learn more, see How Agent Gateway enforces policies.

• Agent Observability: Agent Gateway generates observability telemetry for all agent interactions at the network layer and exports it to Agent Observability to provide you with a comprehensive understanding of agent actions.

Agent Registry

Agent Registry is a centralized catalog that lets you store, discover, and govern agentic components, such as Model Context Protocol (MCP) servers, tools, standalone skills, and AI agents, within Google Cloud.

As you deploy and expand AI capabilities, Agent Registry provides the core infrastructure for your environment, bringing together autonomous agents with the consistent data contexts and tools that MCP servers offer. By consolidating these agents, MCP servers, skills, and endpoints, Agent Registry addresses common challenges in complex AI deployments, such as fragmented tool access, isolated data, and redundant implementations.

Based on the data model that the Agent Registry API uses, you manage Agent, McpServer, Endpoint, Skill, SkillRevision, and Publisher resources to register, search for, and govern agents, endpoints, MCP servers, and skills in Agent Registry.

Why use Agent Registry?

Agent Registry is structured around three primary goals to help you manage your agents, MCP servers, and endpoints:

• Accelerate development: Discover and reuse existing agentic capabilities such as A2A skills, standalone skills, and MCP tools across your organization. You can query the registry to identify and use available capabilities, removing the need to create custom integrations for each new process.

• Simplify integration: Use standardized MCP or Agent2Agent (A2A) protocols to locate and connect to functional endpoints, and help reduce infrastructure management overhead.

• Support governance: Enforce security and implement boundaries across your agentic fleet. Other services can use the identifiers of your agentic components to define permissions and control which agents can access your data, including centralizing agent metadata across multiple Google Cloud projects for governance with Agent Gateway.

• Agent Registry - Documentation

• 
  

Agent Identity 

Agent Identity provides a strongly attested, cryptographic identity for each agent that is based on the SPIFFE standard. With Agent Identity, your agent can securely authenticate to MCP servers, cloud resources, endpoints, and other agents, acting either on its own behalf or on behalf of an end user. Agent Identity uses the agent's own credential and Agent Identity auth manager. You can use the auth manager to create and manage auth providers, which are the specific configurations used to acquire, manage, and secure API keys, OAuth client IDs, OAuth client secrets, and delegated end-user OAuth tokens.

Unlike service accounts, agent identities are not shared by multiple workloads by default, can't be impersonated, and don't allow developers to generate long-lived service account keys. Access tokens generated for Google Cloud are cryptographically bound to the agent's unique X.509 certificates to prevent token theft.

Agent Identity works with Agent Registry and Agent Gateway to help secure and govern AI agents. Agent Identity provisions cryptographic identities and manages credentials, Agent Registry catalogs the tools and destinations in your environment, and Agent Gateway enforces access policies and inspects network traffic when agents call those destinations.

• Agent Identity Documentation 

Agent Gateway

Agent Gateway is the key enforcement component of Agent Platform. It acts as the network entry and exit point for all agentic interactions. It gives enterprise security administrators the ability to secure connectivity for all agentic interactions, whether they occur between users and agents, agents and tools, or among agents themselves.

Key Benefits 

Agent Gateway offers several benefits for both AI developers and enterprise administrators, addressing the complexities of securing and managing AI agent interactions at scale.

For AI developers:

• Simplified innovation: Developers can focus on building agents without managing complex networking primitives or security overhead.

• Protocol mediation: Developers can seamlessly use their choice of agentic protocols such Model Context Protocol (MCP), Agent-to-Agent(A2A), REST, and gRPC while adhering to enterprise security standards.

• Framework agnostic: Functionality is available regardless of the development framework or client used.

• Secure transport and authentication: Automatically handles mTLS handshakes and termination to ensure encrypted connectivity between agents and tools without developer effort. Integrates with Agent Platform's Agent Identity Auth Manager to simplify and secure Oauth 2.0 handshakes between their agents and tools.

Model Armor 

Model Armor is a Google Cloud service designed to enhance the security and safety of your AI applications. It works by proactively screening LLM prompts and responses, protecting against various risks and ensuring responsible AI practices. Whether you are deploying AI in Google Cloud or other cloud providers, Model Armor can help you prevent malicious input, verify content safety, protect sensitive data, maintain compliance, and enforce your AI safety and security policies consistently across your AI applications.