What the Chat Window Leaves Out

Picture a small online shop, made up for this example. A customer types a question into the chat window on its home page. Where is my order? The chatbot answers politely within a second and names a delivery day, and the customer is happy.

The order never shipped. Nobody checked, because nothing behind the chat window could check. The answer sounded right, and that was all it had to do.

That gap is the subject here. Many people picture a chat window when they hear Artificial Intelligence (AI). A chat window is one part of something larger.

What is the difference between an AI system and a chatbot?#

The difference is everything around the chat. A chatbot is a box on a screen, where a question goes in and an answer comes out. A complete AI system wraps that box in the parts that make an answer safe to act on.

A federal agency offers a working definition. The National Institute of Standards and Technology (NIST) describes an AI system as one built to produce outputs such as predictions, recommendations, or decisions (NIST risk framework). It adds that these systems run with varying levels of autonomy.

Read the nouns in that list. Predictions and recommendations are words on a page. Decisions change something in the world, such as money moving, a refund being granted, or an order finally leaving the warehouse on a truck. A chatbot mostly stops at the words. That is the line.

The two are easy to blur. The chat window is what a buyer sees in a demo, so it can become the whole product in the buyer’s head. It is not.

What does a chatbot actually do?#

A chatbot does one step. It reads what you typed and writes a reply. Ask it about a refund and it writes about refunds, fluently and at length, in whatever tone its makers chose for it. It cannot open the refund records unless someone has built that connection. Without one, it guesses.

That single step has a known weakness. NIST also publishes a guide to generative AI, meaning AI that writes new text or pictures. It names the weakness confabulation: content stated with confidence that is wrong or false (generative AI guide). A chatbot alone has nothing to catch it. The reply arrives in the same calm voice either way.

Like a receptionist who answers every phone call but holds no key to the stockroom, a chatbot can be friendly and quick all day. It still cannot tell you what is on the shelf. Somebody has to give it the key, and somebody has to decide which doors it opens. Then it helps.

What are the parts of a complete AI system?#

In this article’s framework, a complete AI system has five parts. Four run in order: information comes in, an AI step drafts the work, a person checks what matters, and an action goes out. The fifth, testing, runs the whole time. In that framework, the chatbot is part two. Here they are, in order.

• Information in. Facts arrive, like an order record.
• AI step. The model drafts an answer.
• Person checks. Someone reviews what matters.
• Action out. The system acts on the work.
• Testing. People test it while it runs.

Order matters as much as the parts. Information comes first because a model can only reason over what it was handed, and a model handed the wrong order record will write a confident answer about the wrong order. The person’s check sits before the action. Never after. An action taken without a check is a guess with consequences.

Testing is the part a user rarely sees. The NIST framework says test, evaluation, verification and validation tasks are performed throughout the AI lifecycle. In plain words, someone keeps checking the system for as long as it runs. One check at launch is not enough.

In this article’s reading, that last part separates a demo from a system. A demo works on the day someone shows it, while a system keeps working in month six, on the questions nobody planned for.

How is an AI agent different from a chatbot?#

Actions make the difference. An agent can search, send, book, or change records on its own. In this article’s five parts, that is part four, the action out, handed to the model itself. The stakes rise with it. A wrong word becomes a wrong deed.

Reach is the new risk. The Open Worldwide Application Security Project (OWASP) is a volunteer group that publishes security guidance for software. Its current list of top risks for AI applications includes one it numbers LLM06 and calls excessive agency. It traces the risk to three causes: too many functions, too many permissions, and too much autonomy.

Its fixes are plain. Give an agent only the tools its job needs, and give each tool only the permissions it needs. Put a person’s approval in front of actions with real impact.

Like a new hire handed the master key on the first morning, an agent with too much reach can do real damage by mistake. Most offices hand out one key at a time. This site has written about an agent like an employee, and the same instinct applies.

A single reply from an agent can also fire many models at once. The more moving parts there are, the more the checks matter.

How do companies describe a complete AI system?#

Companies often describe them in their own words, on pages written to win customers. Two examples follow. Read both pages as claims, written by the people selling the work, and set them against the five parts above.

Both companies are mine. I run them, so the bias runs my way.

The AI Brain describes four steps on its How it works page: find, connect, teach, answer. Find means listing every place a company’s data lives. Connect means linking it, so one customer is one customer everywhere. Teach means tuning the system to how a team decides. Answer means replying with sources and sending alerts when things change.

AI50, the second company, describes stacked services on a shared base. Its services page lists customer facing AI, internal AI, autonomous workflows, and content, all resting on what it calls a smart foundation. For its autonomous workflows, it says a person approves what matters.

Set the two descriptions beside the five parts. Both name information in and an answer or action out, but only one says, in words, where a person approves. The AI Brain page does describe a test before launch, matching the system against the team’s own answers. Neither page is evidence that its system works. A description is a promise, and a promise is checked by testing.

Where do people still decide in an AI system?#

At the points where a mistake would cost something. That is where a person belongs. The NIST framework says human roles in deciding and overseeing AI systems need to be clearly defined. It adds that some systems may not require human oversight, and others specifically do.

That gives any reader a usable test. Three questions. Ask where the information comes from, who checks the work before the system acts, and what happens when the answer is wrong. A system with no clear answer to the second question is still a chatbot, whatever it is called. In my view, the autonomy you can supervise is the real ceiling.

NIST’s generative AI guide makes the same point for organizations. It says their use of these tools may warrant additional human review, tracking and documentation. The same guide names how a check goes soft: automation bias, which it calls excessive deference to automated systems. A person who approves every answer without reading it is no longer checking anything.

Like a smoke alarm that only matters if someone tests the battery, a check nobody runs protects nobody.

Bring it home. A daughter helping her mother with a pharmacy refill might ask a chat window whether the refill went through. The chat window answers in a second, on her phone. The system behind it decides whether that answer is true.

Before she drives over, she can ask the pharmacy one question. Did a person or a record confirm it? The chat is the part you see. The rest is the part that keeps the promise.

Source: National Institute of Standards and Technology, the federal standards agency, Artificial Intelligence Risk Management Framework.