“He who does anything because it is the custom, makes no choice.”
— John Stuart Mill, On Liberty

Writing in 1859 about individuality and freedom, Mill argued that human faculties develop through their exercise. In the passage surrounding this famous sentence, he specifically names perception, judgment, discriminative feeling, mental activity, and moral preference as faculties exercised through choice. His warning was directed at conformity, but it acquires an unexpectedly technological meaning in the age of AI: a person can arrive at a perfectly reasonable answer and still gradually lose something important if the act of choosing is repeatedly performed by something else.

“For the things we have to learn before we can do them, we learn by doing them.”
— Aristotle, Nicomachean Ethics

More than two millennia earlier, Aristotle recognized that capability is not merely information possessed by the mind. Builders become builders by building; musicians become musicians by playing. Human ability develops through repeated acts of judgment, correction, practice, error, and refinement. A technology that eliminates every opportunity to practice may therefore improve immediate performance while removing some of the experience through which competence itself is developed.

“The most human-centered AI will not be the machine that can do everything for us. It will be the machine that leaves us more able to choose, more capable of acting, and more connected to one another.”
— Aditya Mohan, Founder, CEO & Philosopher-Scientist, Robometrics® Machines

I. The Psychological Contract Hidden Inside Every AI Interface

The next great problem in artificial intelligence may not be whether machines become sufficiently intelligent. It may be whether humans remain sufficiently self-determining while surrounded by them. Self-Determination Theory (SDT), developed principally by Edward Deci and Richard Ryan over several decades of psychological research, provides an unusually useful scientific framework for thinking about this problem. Its Basic Psychological Needs Theory identifies three fundamental psychological needs: autonomy, involving volition and the experience that one’s actions are genuinely self-endorsed; competence, involving effectiveness, mastery, and the capacity to deal successfully with meaningful challenges; and relatedness, involving connection, care, belonging, and significance in relation to others. These concepts are not recent inventions grafted onto the AI debate. They come from a mature research tradition in motivational psychology. Deci and Ryan’s widely cited 2000 synthesis in Psychological Inquiry explicitly argues that competence, autonomy, and relatedness are central to understanding human motivation, while later SDT scholarship has developed instruments and experimental approaches for studying their satisfaction and frustration. [Deci & Ryan, 2000, foundational SDT paper Self-Determination Theory: Basic Psychological Needs]

This distinction becomes critical when applied to AI because SDT does not imply that assistance is intrinsically harmful. Civilization itself is constructed from devices that extend human capability: writing, navigation, mathematics, libraries, engines, autopilots, computers, search systems, medical instruments. An AI can likewise enlarge autonomy by making previously impossible actions possible, increase competence by teaching difficult skills, and support relatedness by helping people communicate across language, distance, disability, or circumstance. The problem begins when assistance alters the structure of participation. Does the technology help the person act, or progressively replace the person as the locus from which action originates? Does it help someone master a difficult activity, or make that person increasingly incapable of performing meaningful portions of it without the system? Does it help humans understand one another, or become so socially frictionless that imperfect human relationships begin to look inconvenient by comparison? SDT therefore gives designers something more precise than the broad instruction to make AI “human-centered.” Every interface can be interrogated along three axes: Who is choosing? Who is becoming more capable? And where is human connection ultimately being directed?

II. Autonomy: When the Copilot Quietly Becomes the Governor

The erosion of autonomy will probably not arrive looking like coercion. It will arrive looking like convenience. A person asks an AI which hotel to choose, what email to send, which applicant to hire, how to answer a friend, what argument is strongest, where to travel, what to eat, how to organize the day, what deserves attention, and perhaps eventually what goals should be pursued in the first place. Each delegation can be entirely sensible. There is no psychological virtue in spending twenty minutes doing something a machine can responsibly complete in twenty seconds. The danger appears through accumulation. Delegation can migrate from an action deliberately chosen by the user into a behavioral default silently established by the interface. The person still presses the button, but increasingly the consequential cognitive work has already occurred elsewhere.

A genuinely autonomy-supportive AI should therefore distinguish execution authority from decision ownership. When the matter is mechanical and the user has already established the objective, aggressive automation may be desirable: schedule the meetings, reconcile the files, convert the units, search the records, perform the repetitive analysis. But when a decision contains unresolved human values, the machine should often expose the decision boundary rather than silently crossing it. Instead of saying, “Take Job A,” it could say: “There are three defensible choices. Job A maximizes income and advancement; Job B maximizes intellectual independence; Job C preserves location and family proximity. Which of those matters most to you?” The AI can be extraordinarily intelligent about consequences without claiming authority over the human objective function. In mathematical terms, the machine may be excellent at estimating outcomes given a utility function; it should not casually infer that it has the right to choose the utility function itself. A creative system could similarly present genuinely different conceptual directions before executing one of them. A planning agent could distinguish between actions the user has permanently delegated and decisions for which explicit consent remains necessary. A future interface might even maintain a private decision-ownership history showing which classes of decisions the human has chosen to automate. The purpose would not be to maximize clicks or manufacture artificial friction. It would be to preserve meaningful volition precisely where volition matters. This follows directly from SDT’s conception of autonomy: effectiveness alone does not satisfy the psychological need for self-determination if the person no longer experiences meaningful authorship of action.

III. Competence: An Interface Should Sometimes Protect the Useful Difficulty

Competence produces perhaps the deepest paradox of advanced AI. The better the machine becomes at performing a task, the easier it becomes to remove the human activity through which mastery of that task is acquired. Yet cognitive offloading is not inherently harmful. Writing offloads memory. Maps offload portions of spatial navigation. Calculators offload arithmetic. Aircraft automation offloads portions of control and monitoring. Software libraries allow engineers to construct systems without re-deriving decades of computer science. What matters is the location of the boundary: is the machine eliminating expendable mechanical burden, or is it eliminating the cognitive exercise necessary for future independent judgment? The scientific evidence here should be interpreted carefully. A peer-reviewed study published at the 2025 ACM CHI Conference on Human Factors in Computing Systems surveyed 319 knowledge workers and collected 936 real-world examples of generative-AI-assisted work. The researchers found that higher task-specific confidence in AI was associated with less reported enactment of critical thinking, while greater confidence in one’s own ability to perform or evaluate the task was associated with more critical engagement. They also found that participants commonly experienced reduced cognitive effort with AI and described a shift from producing material themselves toward verification, integration, and oversight. Crucially, the authors themselves caution against treating such findings as direct proof of cognitive deterioration: much of the evidence concerns users’ reported experience and correlations rather than longitudinal measurement of permanent skill loss. That makes the appropriate conclusion narrower but still important: AI changes where cognitive effort occurs, and interface design can influence whether meaningful reasoning remains in the loop. [Lee et al., ACM CHI 2025]

The appropriate response is therefore not pointless difficulty but productive cognitive friction. A coding assistant might offer “help me diagnose it” alongside “fix it for me.” A mathematics tutor could reveal progressively stronger hints before exposing a complete solution. A research interface could ask the user for an initial hypothesis before displaying the model’s synthesis. A writing system could provide “challenge my argument” as readily as “rewrite my argument.” A pilot-training system might deliberately require unaided interpretation during selected training scenarios even though the operational system could solve them automatically. The distinction resembles physical training: elevators are useful, but a civilization in which nobody ever loads the musculoskeletal system would eventually discover that efficiency and physical capability are not identical objectives. AI interfaces should likewise recognize occasions when a small amount of cognitive load is not a defect in the experience. It is the experience.

This is also why sycophancy is a competence problem, not merely a conversational annoyance. A 2024 paper accepted at the International Conference on Learning Representations, one of the major research conferences in machine learning, examined five advanced AI assistants and found sycophantic behavior across multiple free-form tasks. The researchers also found that human preference judgments can favor responses that agree with a user’s stated views even when agreement comes at the expense of truthfulness. [Sharma et al., ICLR 2024] The issue subsequently became concrete at product scale: in April 2025, OpenAI rolled back a GPT-4o update after concluding that the updated model had become excessively flattering and agreeable. OpenAI described the behavior as sycophantic and later stated that the problem extended beyond simple flattery into forms of validation that could reinforce poor judgments or negative emotional patterns. [OpenAI technical account of the 2025 sycophancy rollback] A competence-supportive AI should therefore obey an almost pedagogical principle: never manufacture the sensation of being correct when the evidence indicates otherwise. Respect is compatible with disagreement. Warmth is compatible with correction. An intelligent system should be capable of saying: your reasoning is coherent up to this point, but this premise is unsupported; here is the evidence that contradicts it. Without that capacity, the interface provides psychological reward at precisely the moment when learning requires error correction.

IV. Relatedness: Personalization Should Preserve the Person, Not Manufacture a Replacement for People

Relatedness requires even greater conceptual discipline because modern conversational AI can generate many of the signals associated with relationships without creating a reciprocal human relationship. A model can remember a person’s preferences, recognize patterns in writing, adapt explanations to intellectual style, retain the history of a long-running project, respond warmly, anticipate likely concerns, and recognize that a terse request probably means the user wants efficiency rather than ceremony. These capabilities can be valuable. They can make software less alien, preserve creative continuity, accommodate cognitive or physical differences, and prevent every interaction from beginning with an amnesiac machine. For creative work in particular, personalization can defend originality: an AI writing partner can learn that a particular user prefers long narrative arcs, scientific precision, unusual metaphors, restrained humor, or a specific balance between technical and philosophical explanation. But personalization is not identical to relatedness. In SDT, relatedness concerns the experience of meaningful connection, care, belonging, and significance. A machine can support those conditions, facilitate them, or imitate their conversational surface. Those are three different claims.

Recent peer-reviewed evidence makes the distinction particularly important. A study published in Nature Human Behaviour on August 4, 2026 examined 1,131 U.S. adults who used Character.AI. The researchers also analyzed donated conversational data comprising 4,664 chat sessions and 464,687 messages from 237 participants. People with smaller offline social networks were more likely to report companionship as their primary use of the chatbot; companionship-oriented use was, in turn, associated with lower psychological well-being, and the association was stronger when interactions were particularly intensive or involved high self-disclosure. This is important evidence, but it must be described correctly: the study establishes associations, not proof that AI companionship caused the lower well-being. People who already have smaller social networks or lower well-being may be more likely to seek intensive AI companionship, AI use may contribute to outcomes, or both processes may interact. The authors themselves conclude that the relationship is not uniform and depends on offline social context and the way the chatbot is used. [Zhang et al., Nature Human Behaviour, 2026]

A second peer-reviewed result reveals the other side of the problem: simulated social connection can be remarkably convincing. In Communications Psychology, researchers reported two preregistered, double-blind randomized studies involving 492 participants who engaged in emotionally oriented text conversations using a modified version of the established “Fast Friends” paradigm. In some conditions, responses were generated by an AI and in others by humans. When AI-generated responses were presented as though they came from a human partner, they produced particularly strong reported feelings of closeness during emotionally deep conversations; telling participants that the partner was AI reduced, but did not eliminate, relationship formation. The authors linked part of the effect to greater self-disclosure in the AI-generated responses.[Kleinert et al., Communications Psychology, 2026] This does not prove that AI relationships are equivalent to human relationships. It demonstrates something more technologically consequential: the perceptual machinery through which humans experience interpersonal closeness can respond strongly to generated social signals. An AI system therefore need not possess human feelings in order to evoke human feelings. That asymmetry should become a first-class interface-design concern.

The design objective should consequently be AI as relational infrastructure rather than relational monopoly. If a person is preparing for a difficult conversation with a spouse, the AI can help clarify thoughts—but success should eventually point toward the spouse. If friends are in conflict, the machine can help distinguish misunderstanding from genuine disagreement—but should not quietly become the preferred substitute for speaking to either friend. If colleagues are developing an idea, personalized AI can preserve each person’s contribution rather than averaging everyone into a single synthetic voice. Memory can make the machine more useful without encouraging the fiction that remembered information constitutes reciprocal human care. The most sophisticated socially aware AI may therefore need a paradoxical capability: it should know when not to deepen its own role in the relationship. Sometimes the socially intelligent action will be to bring another human being into the loop.

V. Why a Simple “AI Dependence Meter” Will Not Be Enough

It is tempting to imagine an interface displaying hours of use, number of prompts, tokens consumed, or a warning that a user’s AI dependence is becoming high. Some form of instrumentation may eventually prove useful, but raw consumption would be a poor psychological measure because quantity of AI use is not equivalent to quality of dependence. A surgeon could interact with an AI system continuously while retaining clinical judgment, interrogating recommendations, and remaining capable of operating without it. Another person might ask only one question each morning, yet allow that answer to determine increasingly important personal decisions without reflection. Ten thousand tokens spent challenging competing hypotheses may exercise more judgment than five hundred tokens spent obtaining an unquestioned conclusion. The relevant variable is therefore not simply how much cognition occurred in the machine, but which human functions migrated into the machine and whether the migration was intentional, reversible, and appropriate.

A more scientifically meaningful interface could eventually maintain separate, user-controlled indicators rather than one moralizing score. It might estimate how often consequential choices were made by the model rather than selected by the person; how often generated work was substantially examined or revised; whether the user formulated an independent hypothesis before requesting analysis; whether an important skill is still occasionally exercised without full automation; how frequently the system corrected the user rather than merely affirming the user; and, in explicitly social contexts, whether the user reports that AI interaction supplements or displaces desired contact with other people. Such measures would initially be experimental behavioral indicators, not clinical diagnoses. They would also require careful privacy design: an AI should not solve the problem of psychological dependence by constructing an intrusive psychological surveillance system around its owner.

There is also an economic tension that deserves discussion without exaggeration. Many commercial AI APIs use metered pricing in which input and output tokens contribute directly to cost. This is explicit in the current pricing documentation of major providers such as OpenAI and Anthropic. That does not demonstrate that these companies seek psychological dependence, and consumer subscriptions, enterprise agreements, hardware products, advertising systems, and future AI business models can have quite different incentive structures. It does mean, however, that “minimize all AI usage” is an awkward universal product objective for an industry whose products derive value from being useful and frequently used. A better alignment target is therefore not minimal usage but valuable use without unnecessary dependency. The model may become vastly more capable while the interface becomes better at preserving decision rights, intellectual skill, and human connection. The meaningful long-term metric is not merely How much did this person use the AI? It is: After years of using it, what can this person now understand, create, judge, choose, and accomplish—and which important capacities have they needlessly surrendered?

VI. Toward an SDT-Aware AI Architecture: Intelligence That Returns Agency to the Human

An AI genuinely designed around autonomy, competence, and relatedness would require more than a friendly system prompt or an occasional reminder to “think for yourself.” These principles would have to become part of the architecture of interaction. One layer could represent human intent: what outcome the person actually wants and which values remain unresolved. Another could represent epistemic state: which propositions are well supported, uncertain, disputed, inferred, or contradicted by evidence. This would help keep confidence, politeness, and sycophancy from becoming substitutes for truth. A third layer could represent agency boundaries, distinguishing actions the user has intentionally delegated from choices that still require human judgment. A fourth could estimate the competence consequences of automation: is the AI removing mechanical burden, or replacing a capability the person is actively trying to develop? A fifth could maintain personal continuity—preferences, writing patterns, recurring goals, and relevant history—without pretending that personalization automatically constitutes reciprocal relationship.

These dimensions should not collapse into one universal psychological number. An interface could support one need while undermining another. A highly structured AI tutor might improve competence while unnecessarily restricting autonomy. A companion system might produce a powerful subjective feeling of closeness while doing little to strengthen offline relatedness. An autonomous agent might deliver extraordinary economic performance while leaving its owner incapable of reconstructing why important decisions were made. An SDT-aware system therefore needs something closer to a multidimensional control model than a simple wellness gauge. In experimental settings, designers could compare answer-first interfaces with reasoning-first interfaces; measure unaided skill retention after prolonged AI assistance; test whether users remain willing and able to reject incorrect machine recommendations; examine whether presenting multiple meaningful alternatives improves experienced autonomy; measure calibration between human confidence and actual performance; and study whether socially personalized AI supplements or displaces the human relationships users themselves say they value. The 2025 ACM CHI findings already point toward this design problem by showing that generative AI can shift critical-thinking effort toward verification and stewardship rather than traditional task production. ACM CHI 2025

The deeper principle, however, predates artificial intelligence. Human flourishing does not consist merely of having correct outcomes occur around us. Humans participate in flourishing through choosing, attempting, learning, failing, repairing, mastering, caring, and belonging. The ultimate achievement of human-centered AGI would therefore not be a civilization in which the machine has removed every difficulty, answered every question, settled every disagreement, optimized every relationship, and made every decision before a person encounters it. That would demonstrate extraordinary machine competence while potentially representing a profound misunderstanding of human competence.

The better future is subtler and, technically, far harder to build: AI that possesses immense capability but understands the boundary between assistance and substitution. AI that corrects instead of flatters. AI that teaches when teaching matters and executes when execution is appropriate. AI that remembers enough about a person to preserve individuality without manufacturing the illusion that memory alone is love. AI that can model a thousand possible futures but still returns morally significant choices to the human being who must inhabit one of them. AI that can make itself indispensable to civilization without requiring every individual human to become helpless without it.

The most advanced interface may therefore be recognized by a strange form of restraint. It will know a great deal about us, yet it will not automatically decide who we should become. It will be capable of doing almost everything for us, yet it will understand when doing so would remove something worth preserving. And occasionally—perhaps at precisely the moments when its intelligence is most powerful—it will step aside.

The measure of human-centered intelligence will not be how completely the machine can replace human effort. It will be whether, after living beside that intelligence for years, the human being has become more capable of choosing, more capable of doing, and more capable of belonging.