// the short answer · updated 2026-08-12

Is it actually an AI agent?

An AI product is an agent when it can pursue a goal in a loop, choose actions and tools, change external state, recover from errors, and continue without constant human supervision.

A chatbot that only answers, a fixed workflow that follows steps someone wired in advance, or a scheduler that fires a prompt is not automatically an agent. The label describes runtime behaviour—not a model, a UI, or a marketing category.

Score one yourself → · Read the exact rules and limitations →

01

The five-question test

yes / no / uncertain

01 · Does it loop until the goal is done?

Yes: it decides how many steps the task needs and continues until done or stuck. No: every run follows a fixed A→B→C flow.

02 · Does it choose tools and actions at runtime?

Yes: it chooses what to do next from what it discovered. No: a person preselected every tool and its order.

03 · Does it act outside the conversation?

Yes: it writes a file, runs a command, submits a form, sends a message, or opens a pull request. Producing advice for a human to act on is not acting.

04 · Does it recover from errors?

Yes: after a failure it works out a different attempt and continues without asking a person. Blindly repeating the same call does not count.

05 · Can it finish without step-by-step supervision?

Yes: a documented mode can complete a task without approval for every action. We disclose when that mode is not the default.

Our deliberately simple bands: 4–5 = agent · 2–3 = wrapper with ambition · 0–1 = cron job with extra steps. The raw answers matter more than the nickname.

02

Agent, chatbot, workflow, or wrapper?

Chatbot

Turns messages into responses. It may reason impressively, but if the human must execute every next step, it does not act as an agent.

Workflow

Automates a sequence chosen by its builder. Branches and retries can make it powerful without giving it control over what the sequence should be.

AI wrapper

Puts a useful product around a model. “Wrapper” is not an insult; it becomes misleading only when a fixed interaction is sold as autonomous agency.

AI agent

Owns the next-step decision inside bounded permissions. Autonomy is not unlimited access: a carefully sandboxed coding agent can still be an agent.

03

Concrete examples

dated dossiers, not vibes

Claude Code and Codex pass because they can inspect a codebase, choose tools, edit files, run commands, and recover inside a task. Operator and Manus are browser/general-purpose examples.

ChatGPT Tasks does not become an agent merely by running prompts on a schedule. Frameworks require care: compare CrewAI, which ships a loop, with LangGraph, which primarily ships primitives for developers to assemble.

Every dossier shows all five decisions, its evidence grade, date, sources, revisions, and a correction path. Browse all 30 verdicts or download the open dataset.

04

What this test cannot tell you

  1. Agency is not quality. A bad autonomous product can score higher than an excellent focused tool.
  2. The threshold is our editorial choice. There is no scientific boundary at four yeses.
  3. Documentation and products change. Treat every result as a dated snapshot.
  4. Recovery is hard to verify from marketing pages. Execution traces are stronger than vendor claims.
  5. Permissions change behaviour. We separate capability from the default safety posture whenever the evidence allows it.

For definitions, edge cases, proof grades, and every known weakness, read the full method. For machine-readable criteria and sources, use the dataset.