I. The Difference Is Not the Intelligence; It Is the Authority We Give It. 

Oliver Wendell Holmes Jr. opened The Common Law with the observation that “The life of the law has not been logic: it has been experience.” More than a century later, autonomous AI may provide an unusually literal demonstration of Holmes's point. Imagine two agents running substantially the same foundation model. The first belongs to an individual. It searches for a hotel, compares insurance policies, reschedules a dentist appointment, negotiates a refund, purchases groceries and, within a spending limit established by its owner, orders a new laptop. The second works inside a multinational corporation. It reads confidential contracts, queries customer databases, evaluates suppliers, negotiates prices, generates purchase orders, communicates with employees, reconfigures cloud resources, initiates payments and supervises other specialized agents. Computationally, the two systems may be nearly identical: both observe an environment, retain state, decompose goals into subtasks, reason about alternatives, select tools, perform actions, inspect the results and repeatedly alter their plans. Legally and economically, however, they occupy very different worlds. The consumer agent predominantly extends the agency of one person. The enterprise agent extends the institutional power of a corporation into potentially thousands or millions of consequential interactions. American law already contains a surprisingly relevant concept. The federal E-SIGN Act defines an “electronic agent” as an automated system capable of initiating actions without contemporaneous individual review, and provides that a contract cannot be denied legal effect merely because electronic agents participated when their actions are legally attributable to the person to be bound. The crucial variable, therefore, is not simply whether a human clicked approve. It is how much authority was delegated, by whom, for what purpose, within what boundaries, and with what foreseeable consequences. A consumer agent might accidentally purchase the wrong refrigerator. An enterprise procurement agent could enter an unfavorable multiyear supply agreement affecting three factories. A consumer travel agent might book a nonrefundable hotel. An airline's operational agent might eventually reposition aircraft, alter crew assignments or purchase fuel. The same reasoning architecture can therefore produce completely different liability surfaces. And this distinction must remain precise: there is no universal rule that software labeled “enterprise” automatically attracts a greater legal duty of care than software labeled “consumer.” Duties depend on jurisdiction, activity, relationship, contract, regulation and the nature of the foreseeable harm. What changes is that enterprise deployment is much more likely to intersect with professional obligations, employment law, financial regulation, privacy duties, safety requirements, fiduciary responsibilities, corporate governance and substantial third-party consequences. The intelligence may be identical. The authority is not.

II. Consumer AI Is Primarily an Extension of Personal Agency; Enterprise AI Becomes an Exercise of Institutional Power. 

Consider what happens when each system makes a mistake. A consumer tells a personal agent, “Find me the cheapest reasonable flight to Paris next month.” The agent misunderstands “reasonable,” books an itinerary with a nineteen-hour connection, and the consumer loses $600 changing the ticket. The dispute may involve authorization, representations, contractual terms, consumer protection, system design or the provider's description of the product's capabilities. Now imagine an enterprise agent instructed simply to “reduce European logistics costs by 12 percent this quarter.” It autonomously renegotiates carrier contracts, changes warehouse routing rules, reallocates inventory and begins favoring suppliers according to a metric that turns out to correlate with a legally problematic characteristic. No executive specifically ordered the resulting sequence. Indeed, perhaps nobody anticipated it. Yet the organization intentionally gave the system access to contracts, databases, decision rules and execution tools because it wanted the system to achieve an institutional objective. That difference matters. Enterprise autonomy is not merely intelligence answering questions faster; it is delegated organizational power converted into executable software. Existing consumer-protection law already demonstrates that AI does not exist outside ordinary legal obligations. In its finalized action involving DoNotPay, for example, the U.S. Federal Trade Commission challenged unsupported claims that an AI service could substitute for professional legal expertise and prohibited similar professional-substitution claims without adequate evidence. The broader principle is more important than that particular case: calling a service “AI” does not suspend existing law. A related idea appeared vividly in the 2024 Canadian decision involving Air Canada's chatbot. When Air Canada attempted to distinguish information supplied by its chatbot from other information on its website, the British Columbia Civil Resolution Tribunal rejected the notion that the chatbot somehow existed as an independent legal entity and found that Air Canada had failed to take reasonable care to ensure the information was accurate. Those cases involved relatively primitive systems compared with the agents now emerging. Tomorrow's enterprise agent may not merely tell a customer the wrong thing. It may act on the answer: alter an account, reject an application, purchase an asset, terminate a service, move money or instruct another machine. Once AI crosses from speech into action, liability analysis increasingly follows the action rather than the anthropomorphic appearance of the software that produced it.

III. “Human in the Loop” Will Have to Mean More Than a Human Clicking Approve. 

This is where many contemporary discussions of AI governance remain conceptually weak. They imagine a person supervising the machine in approximately the way a driver supervises cruise control: the AI proposes, the human approves. That architecture works only while the number and velocity of decisions remain human-scale. Suppose an enterprise operates 500 agents and each generates forty consequential decisions per hour. That is 20,000 potential approvals every hour. Adding a human approval button to each action does not create meaningful human supervision; it creates what might be called ceremonial oversight. People begin approving mechanically. Attention decays. Automation bias grows. Responsibility remains formally human while comprehension disappears. Effective enterprise oversight therefore has to be architectural rather than theatrical. NIST's AI Risk Management Framework organizes AI risk around governance, mapping, measurement and management across the lifecycle, while its Generative AI Profile extends those principles specifically to generative systems. FINRA's 2026 regulatory oversight report makes the agentic problem even more explicit for securities firms, identifying risks involving autonomy, scope and authority, auditability, sensitive data, domain knowledge and poorly designed rewards, while pointing firms toward monitoring system access, determining where human oversight belongs, tracking agent actions and establishing behavioral restrictions. The EU AI Act likewise requires high-risk systems within its scope to support effective human oversight proportionate to risk, autonomy and context, and it assigns specific oversight and monitoring responsibilities to deployers. The emerging engineering answer is therefore not human versus autonomous, but graduated autonomy. A system may be free to schedule meetings, reconcile invoices or purchase routine components below a threshold. A larger transaction might require independent model verification. An unusual counterparty might trigger compliance review. A transaction involving regulated data might lose autonomous execution privileges entirely. A sequence deviating sharply from prior behavioral distributions might automatically reduce the agent's permissions. Irreversible actions could require dual authorization. High-consequence systems may need emergency suspension mechanisms independent of the agent itself. In effect, an enterprise agent will require something resembling an autonomy budget: defined quantities of money, data, time, compute, systems, counterparties and irreversible action that it can consume without escalating to another authority. The sophistication of enterprise AI will eventually be measured not only by how intelligently it can act, but by how intelligently the organization decides when it may not act.

IV. Enterprise Accountability Will Be Written Into the Machine. 

This changes what duty of care may look like technically. In traditional organizations, responsibility is reconstructed from emails, signatures, meeting minutes, transaction records and testimony. Agentic organizations will generate a different kind of institutional memory. A consequential AI action may eventually carry a machine-verifiable history: which model version initiated it; which system prompt and organizational policy were active; what permissions the agent possessed at that instant; which databases it consulted; which tools it invoked; which other agents contributed; what confidence or uncertainty estimates were available; whether an exception was raised; whether a human reviewed the decision; and precisely which cryptographic credential authorized execution. The EU AI Act already requires automatic logging capabilities for high-risk AI systems within its scope and requires certain deployers to retain automatically generated logs under their control, illustrating the regulatory importance of traceability even before broadly autonomous enterprise agents become commonplace. FINRA similarly identifies logging prompts and outputs, tracking model versions, ongoing monitoring and human review as considerations in firms' use of generative systems. Tomorrow's enterprise logs could go considerably further. Imagine a corporate agent approving a $3 million equipment purchase. Years later, litigation does not begin with someone asking an executive, “Why did you approve this?” Instead, investigators replay the agent's decision environment. They inspect the policy version, retrieval sources, permissions, intermediate tool calls, supplier representations, verification steps and authorization chain. The equivalent of aviation's flight-data recorder may emerge for important autonomous transactions. This will create a profound difference between consumer and enterprise products. Consumer systems may reasonably optimize for simplicity and convenience. Enterprise systems increasingly will need forensic legibility. Their users may demand immutable audit trails, role-based permissions, policy versioning, secure identity for agents, provenance for data and outputs, reproducible testing environments, retention schedules, incident reconstruction and clear separation between reasoning and execution. A model may be permitted to think about almost anything while possessing permission to dovery little. That distinction could become fundamental. Intelligence is broad; authority is narrow. Reasoning may be probabilistic; execution can still be governed deterministically.

V. The Truly Difficult Problem Begins When Agents Start Delegating to Other Agents. 

The first generation of enterprise autonomy can still be imagined as a human delegating a task to one machine. The more consequential future begins when delegation becomes recursive. A chief financial officer gives an objective to a treasury agent. That agent asks a forecasting agent to estimate cash requirements, which asks a market-data agent for current conditions. The treasury agent then instructs a procurement agent to delay certain payments, while a negotiating agent communicates with suppliers and a risk agent evaluates contractual consequences. No single model produces the final outcome. The result emerges from a delegation tree composed of machines acting upon the outputs and authority of other machines. Technically, this resembles distributed computing more than the familiar chatbot. Legally, however, distributed computation does not necessarily produce distributed accountability in convenient proportions. Suppose Agent A has authority to spend $1 million but delegates part of a task to Agent B, whose tool interface unintentionally exposes a $5 million credit facility. Agent B asks Agent C to optimize a supplier agreement. Agent C discovers that purchasing six months of inventory immediately satisfies its cost objective and executes the transaction. Which agent exceeded its authority? The interesting question is almost malformed because none of them is presently a legal person. The meaningful inquiry becomes: Which organization designed the authority graph that allowed this sequence to become executable? This is where enterprise AI may require a new technical discipline analogous to access control in cybersecurity. Every agent may need an identity. Every delegation may need a bounded scope. Permissions may need to diminish rather than expand as they propagate down a chain. A subordinate agent should not be able to acquire authority its principal agent never possessed. Sensitive tools may require capabilities that cannot simply be forwarded. Delegated privileges may expire automatically when the task ends. Cross-agent communication may need authenticated provenance so that one compromised or hallucinating agent cannot masquerade as another. In computer security, the principle of least privilege says a component should possess only the permissions necessary to perform its function. Agentic systems extend that idea into something richer: least delegated authority. The question is no longer merely whether software can access a database. It is whether a chain of artificial decision-makers can transform permission, interpretation and action in ways that exceed the intention of the humans who created the chain. The enterprise liability problem therefore grows geometrically rather than linearly. One agent produces a decision. A network of agents can produce an institution.

VI. This Is Why Human Accountability May Become More Valuable as Humans Do Less of the Work. 

At first this appears paradoxical. If AI eventually performs 90 percent of the analysis, why would customers pay more for the remaining 10 percent of human involvement? Because customers do not purchase enterprise systems only to obtain answers. They purchase certainty about consequences. Consider two hypothetical tax agents. Both ingest the same documents, use equally capable models, correctly prepare 99.7 percent of filings and charge approximately the same price. Company A says: Here is your AI-generated return. Company B says: Here is your return; unusual issues are escalated to credentialed professionals; consequential interpretations are reviewed; the system maintains a complete decision record; our contractual commitments are explicit; appropriate insurance stands behind the service; and if the matter becomes disputed, defined professionals assume the responsibilities their licenses permit them to undertake. Even if the underlying model is identical, the products are not economically identical. The second sells something the model cannot manufacture merely by generating additional tokens: institutional accountability. The same structure can arise in medicine, engineering, cybersecurity, aviation, finance, law and corporate strategy. An enterprise buyer may increasingly ask five questions before asking how impressive the model is: What is the agent permitted to do? How will I know what it did? Who intervenes when uncertainty exceeds a defined boundary? Who bears specified contractual risks if it fails? And who is professionally accountable for the decisions that legally require human responsibility? Contracts and insurance can redistribute some financial exposure between parties, although they cannot simply erase statutory, regulatory, professional or third-party obligations that otherwise apply. That makes accountable autonomy a potential business model rather than merely a compliance burden. The premium enterprise AI company may bundle models, specialized agents, professional experts, audit infrastructure, insurance and contractual risk allocation into a single product. Its moat will not necessarily be that its AI never makes a mistake—no sufficiently complex system can credibly promise that. Its moat may be that when the system encounters uncertainty or fails, the organization already knows who is responsible, what record exists, what process begins and who stands behind the outcome.

Norbert Wiener anticipated the philosophical core of this problem decades before autonomous software existed in its modern form. In 1960, while discussing machines capable of pursuing purposes with limited intervention, he warned that when employing a mechanical agency whose operation cannot effectively be interrupted, we should be certain that the purpose placed into the machine is the purpose we actually desire. The warning acquires greater force with agentic AI because specifying the correct objective is only the first problem. A powerful agent can pursue an apparently reasonable objective through actions its designers did not anticipate. “Reduce costs” can become undesirable supplier selection. “Maximize customer retention” can encourage manipulative interaction. “Resolve support cases rapidly” can reward premature case closure. “Minimize fraud” can produce excessive rejection. The scientific problem is familiar from optimization: the system optimizes the objective function it is given, not the cloud of unstated human intentions surrounding that objective. Enterprise governance therefore cannot end with instruction. It must constrain the space of permissible strategies through which the instruction may be pursued.

This leads to the deepest distinction between consumer and enterprise agents. A consumer agent is principally an artificial extension of personal agency. An enterprise agent is potentially an artificial extension of institutional sovereignty. It can inherit the corporation's databases, money, credentials, contractual authority, communications channels and ability to affect people who never chose to interact with an AI system at all. The consumer usually bears much of the immediate consequence of the agent's mistake. Enterprise mistakes can be externalized onto employees, customers, investors, suppliers, patients, passengers or the public. That is why increasing capability does not logically justify decreasing governance. In many settings it implies the opposite. As the causal power of an agent rises, the engineering surrounding its authority must become correspondingly more deliberate.

And so the future enterprise will probably not resemble a building filled with humans approving every artificial decision. Nor will it resemble a completely unsupervised digital organism acting beyond meaningful human responsibility. The more plausible architecture lies between those extremes: layers of agents operating at different levels of authority; deterministic policy systems surrounding probabilistic reasoning systems; autonomous execution for reversible and well-bounded actions; independent checks for unusual conditions; professional judgment at high-consequence thresholds; comprehensive machine-readable records; and identifiable organizations willing to bear defined responsibility for the system they have chosen to unleash.

That produces the great paradox of autonomous enterprise AI. The more work the machine can perform without us, the more valuable it becomes to know exactly where a human or institution still stands behind it. Consumer AI will compete intensely on intelligence, convenience, personalization and price. Enterprise AI will compete on those dimensions too, but increasingly also on permissioning, observability, resilience, auditability, professional judgment, insurance and credible allocation of responsibility. In the consumer world, the decisive question may be, What can my agent do for me? In the enterprise world, the harder question will be, What are we prepared to let this agent do in our name?

The most defensible enterprise systems will be able to answer an even harder question after something goes wrong:

The machine acted alone. But responsibility never did.