How Much AI Autonomy Can Your Team Actually Supervise? · Hanh D. Brown

Every Artificial Intelligence (AI) tool you use has a dial on it. Some show it plainly. Most hide it inside a product decision somebody made without telling you. The dial runs from a suggestion you can ignore to a machine that goes off and does the whole job.

Most of the argument about agents is really an argument about where that dial should sit.

What is an autonomy slider in an AI product?#

A dial, not a switch. That distinction matters more than it sounds. Most product arguments treat automation as on or off. Almost nothing real works that way.

Take a coding tool. At the lowest notch it finishes the word you are typing. Nudge it up and it rewrites the block you selected. Higher still, a whole file. At the top it works across the entire repository while you make coffee.

That same dial exists in search tools, in writing tools, in customer service tools. A quick lookup sits at one end. A long unattended research run sits at the other. Same dial.

Here is what a good product does with it. It gives the dial to the user, and lets it move per task rather than per subscription. That is a design choice. It is also where a lot of tools quietly fail the people using them.

Why is the human the bottleneck in an AI workflow?#

Somebody has to check the work. I have watched a whole meeting go by where everyone argued about which model to buy, and nobody asked who would read what it produced. That one sentence carries this whole article. It survives every improvement in the models, because a mistake nobody caught is still a mistake that shipped.

Models are strange colleagues. They hold more facts than any person you have met, then fail at something a child would catch. They forget everything between sessions, and they believe whatever the last convincing text told them.

Like an understudy who knows every line but has never seen the set, the machine needs its context handed over fresh each morning. Nothing consolidates. What was obvious to it on Tuesday is gone by Wednesday, and that pattern shows up in what is still missing on the road to general intelligence.

So the throughput limit is not generation. It is verification. A machine that produces faster than you can check has not saved you time. It has moved your work.

Which leads to the practical rule, and it is a small one. Keep the changes small enough to audit. Nobody accepts a ten thousand line change on trust. They accept it on exhaustion.

How do you set the right autonomy level for a task?#

Two questions decide it, and neither one is the benchmark score or how impressive the demo looked. How hard is this to undo? How fast can a person tell whether it went right?

Cross those two and the answer falls out. Work that is easy to undo and quick to check can run on its own. The worst case is a wasted afternoon. Work that is hard to undo and slow to check needs a person before it happens, not after.

The mixed corners are where judgment lives. Hard to undo but quick to check means you watch it closely with a hand near the switch. Easy to undo but slow to check means you let it run and sample the output.

Notice what is missing from all four boxes. The model’s intelligence never appears. The setting depends on the shape of the work, not the cleverness of the tool. So a smarter model does not automatically earn a higher notch.

Regulators reached the same structure from a different road. The National Highway Traffic Safety Administration (NHTSA) describes vehicle automation in graded levels rather than as a switch (automated driving systems). The handoff is the hard part.

Why is this the decade of agents, not the year of agents?#

The last mile is long. Every few months something arrives that looks finished. It is not. Then the unglamorous work of making it reliable takes years.

Driving is the honest comparison. A flawless demonstration ride happened more than a decade ago. Today there is still a person in the loop, in the driver seat or at a screen in a remote operations room. The demo was never the hard part.

Guidance for partly automated vehicles is blunt about why. When the car does most of the driving, the driver disengages. Getting a disengaged person back at the moment it matters is its own engineering problem (human factors guidance).

The same problem is arriving in software, in different clothes. An agent that is right most of the time trains you to stop reading its output. Meanwhile the way AI news gets reported rarely separates the demo from the decade.

How do you make your website readable by an AI agent?#

Write for the machine as well as the person. A new kind of reader is arriving on your site, and it does not have eyes, patience, or a mouse.

Four changes cover most of it.

• Plain text. Publish a markdown version of your documentation alongside the pretty one.
• An index file. A short list at a known address telling an agent what is here.
• Commands, not clicks. Replace click the settings tab with a command an agent can run.
• A shared protocol. Adopt the standard connection method agent tools already speak, so each one does not need a custom build.

None of that is exotic. It is the same instinct that made pages readable to search engines when that audience first mattered. The audience changed. The discipline did not.

There is a payoff worth naming. A tool that can read your product correctly will recommend it correctly, and this is a slow shift in what computing itself is becoming.

What should you automate first?#

Start where a mistake is cheap and obvious. That is the whole rule. Nothing more. It sounds too simple until you watch a team do the opposite, and automate the highest value process in the building on day one.

Cheap and obvious means two things at once. If the work goes wrong you can undo it, and you will find out quickly rather than in a quarterly report.

Like a swimmer who starts in the shallow end rather than the open lake, you are not testing the water. You are building your own ability to read it.

Then move the dial up one notch at a time, letting the evidence decide the pace rather than the marketing. Each notch earns the next.

It settles at my own house. When my daughter first cooked dinner alone, I did not leave the building. I sat in the next room with the door open. Close enough to hear the smoke alarm. Far enough that the dinner was really hers.

That is the whole design problem. Give the work away, stay near enough to catch it, and move further out only when the evidence says you can.

Source: Andrej Karpathy, an AI researcher and the former Director of AI at Tesla, in a conference talk.