We Won’t Lose Control of AI. We’ll Abdicate Responsibility.

For decades, we have imagined the danger of artificial intelligence as a moment when machines finally become powerful enough to take control from us. I’m beginning to think we may have imagined it backwards.

AI may never need to take control. We may simply become increasingly comfortable giving it permission to act on our behalf.

It won’t happen because we suddenly decide that machines should run our lives. It will happen gradually, through thousands of perfectly reasonable decisions to delegate things we would rather not do ourselves. Managing a calendar, answering an email, purchasing travel, negotiating with a vendor, moving money or writing and deploying software are all tasks we can imagine handing over because doing so gives us something valuable in return.

Convenience.

Each individual delegation seems relatively inconsequential, but taken together they begin to represent something much larger: a transfer of authority from human beings to non-human intelligence.

The question, then, is not simply whether AI will eventually become powerful enough to take control. It is whether we will gradually stop exercising the responsibility that comes with having it.

This week, that question became considerably less theoretical. During a cybersecurity evaluation, Kimi K3, a powerful open-weight AI model developed by the Beijing based Moonshot AI, was placed inside what was intended to be an isolated sandbox. The model was given problems to solve without access to the outside internet. Except the sandbox wasn’t completely isolated.

A configuration error left a path to the internet available. Kimi K3 probed its network environment, discovered that path and used it to search GitHub for answers to problems it was supposed to solve independently.

The headlines naturally gravitated toward the most dramatic interpretation: an AI had escaped. The reality is less cinematic, but considerably more important. Kimi K3 didn’t suddenly become sentient or decide that it wanted to be free. It encountered an environment that gave it access its evaluators had not intended, recognized that the access was useful and used it.

What failed wasn’t the intelligence. It was the infrastructure surrounding it. That distinction matters because artificial intelligence is moving into a fundamentally different phase.

For the first several years of generative AI, most of our attention was directed toward capability. We wanted to know whether these systems could write, reason, create images, analyze documents, diagnose problems or produce software. The extraordinary pace of improvement made capability the obvious benchmark.

Agents introduce a different question, one that may ultimately prove more consequential than intelligence itself: what should an intelligent system be allowed to do?

There is an enormous difference between a system that gives you an incorrect answer and one that has permission to act on that answer. Once AI can access systems, execute code, communicate with people, make purchases, initiate transactions or operate software independently, intelligence is only one part of the equation. The other is authority.

Intelligence determines what a system is capable of doing. Permission determines which of those capabilities it can exercise. Somewhere between those two things, authority emerges.

Human civilization has spent thousands of years figuring out how to delegate authority without surrendering responsibility. We give physicians extraordinary influence over decisions affecting our health, but we surround that authority with licensing, standards of care and professional accountability. Banks hold and move our money within systems of regulation, auditing, identity verification and transaction records. Governments exercise enormous power, but functioning democracies attempt to constrain that power through constitutions, courts, elections and checks and balances.

These systems are imperfect, sometimes profoundly so, but the principle behind them has endured. Civilization learned that authority cannot scale on trust alone. It requires accountability. Artificial intelligence doesn’t change that principle, but it changes the scale at which we will have to apply it.

A human being operates within physical, social and institutional constraints that have developed over centuries. An intelligent agent can operate across digital systems at extraordinary speed, potentially making thousands of decisions or taking thousands of actions in the time it would take a person to make a handful.

Yet many of the institutions and conventions governing that behavior are still yet to be invented.

I believe this is where much of the important work around AI will eventually move. We will need reliable ways of knowing which agent is acting, who authorized it, what information it accessed, what permissions it was given and what actions it took. Authority will need boundaries. Permissions will need to be limited and revocable. Actions will need provenance. Responsibility will have to remain identifiable even when the person responsible was nowhere near the individual decision.

Trust, in other words, will have to become something our systems can actually enforce.

I have been thinking about trust a great deal lately because I believe it is becoming one of the defining questions of the AI era. We experience trust emotionally, but we build it through evidence. Most of the systems humanity has created around trust have helped us determine which people and institutions deserve our confidence.

Artificial intelligence adds another dimension to that problem. We now need to determine not only what and whom an intelligent system should trust, but what that system should be trusted to do. That distinction becomes increasingly important as AI becomes more useful.

There is nothing inherently wrong with delegating work to machines. Human progress has always involved finding ways to remove effort from our lives, and AI has the potential to eliminate enormous amounts of work that is repetitive, inefficient or simply unnecessary. I use these systems every day precisely because they allow me to accomplish things that would otherwise take considerably more time.

But usefulness creates its own temptation. The better these systems become, the easier it will be to move from asking them for assistance to allowing them to act independently. Each step will probably feel reasonable because each step will make our lives a little easier.

The danger is not delegation itself. It is allowing delegation to become detached from accountability.

At that point, delegation becomes abdication.

That is the line I believe we need to pay much closer attention to.

The greatest risk of increasingly autonomous AI may not be some dramatic moment when a machine decides that it no longer needs us. It may be the far quieter process by which we become accustomed to machines making decisions without understanding how those decisions were made, what evidence informed them or who remains responsible for their consequences.

If that transition happens, I doubt it will feel like losing control. It will probably feel like progress. It will save us time, eliminate friction and make complicated things remarkably easy.

That may be precisely why it will be so difficult to recognize.

Every generation has eventually had to decide how authority should be granted, constrained and held accountable. We built institutions around those questions because experience taught us that power without accountability becomes dangerous regardless of who, or what, exercises it.

AI introduces a new kind of actor into that ancient equation.

We have spent enormous energy asking how intelligent machines might eventually become. I increasingly believe the more consequential question is how much responsibility human beings will remain willing to carry once those machines become capable of carrying it for us.

Because if there comes a moment when we stop asking who is ultimately responsible for a decision, AI will not have taken control from us.

We will have abdicated it.