AuxeonChoose an AI specialist
Menu
← All insights

How to evaluate an AI agency before you sign

Many vendors now describe their products as agents. Six questions separate the systems that will carry real work from the ones that won’t.

Share on LinkedIn ↗
Six glass evidence panels show a workflow map, decision routing, a measurement plan, a runbook, linked records and a scope boundary. Autonomy is limited to approved scope; consequential steps require the named human approver.
Autonomy is limited to approved scope; consequential steps require the named human approver.

The market for AI services is crowded, and much of it has been relabeled. In June 2025, Gartner described a practice it called "agent washing": rebranding existing assistants, chatbots, and automation tools as agentic without substantial new capability. It estimated that only about 130 of the thousands of vendors claiming agentic AI were real.1

Regulators have noticed as well. In September 2024, the Federal Trade Commission announced Operation AI Comply, a set of law enforcement actions against companies using AI claims to deceive customers.2

For a business choosing a partner, the question is not whether a vendor uses AI. Nearly all of them do. The question is whether what they build will hold up inside your business. These are the questions we would ask.

Conceptual operating record

A six-part test for an AI agency

Diagram of six questions for evaluating an AI agency: map the work, locate human decisions, define baseline and ownership, plan post-launch monitoring, retain inputs and approvals, and state explicit limits.

  1. MAP

    Workflow; Friction; Intended measure;

  2. DECIDE

    Autonomous; Escalated; Human-approved;

  3. MEASURE

    Baseline; Metric; Named owner;

  4. OPERATE

    Monitoring; Exceptions; Change approval;

  5. RECORD

    Inputs; Outputs; Decisions; Approvals;

  6. BOUND

    In scope; Out of scope; No guaranteed outcome;

A credible answer names the evidence, owner and limit.

1. What do you do before you build?

A vendor that proposes a tool in the first meeting is selling what it already has. The work that decides whether AI pays off, mapping how work moves, where it stalls, and what should be measured, comes before any build. Ask to see what that phase produces.

2. Where does a person decide?

Every credible system has limits. Ask the vendor to show exactly which actions the system takes on its own, which it escalates, and who approves consequential steps. If the answer is "the AI handles it," keep asking.

3. How will we know it worked?

Ask what will be measured, against what baseline, and who owns the number. MIT’s research on generative AI pilots found that most produced no measurable change in profit and loss.3 Measurement agreed at the start is the difference between a result and an anecdote.

4. What happens after launch?

Models change, processes change, and the business grows. Ask who monitors the system, who handles exceptions, and how changes are tested and approved. A system without an owner after launch begins to degrade on day one.

5. Who owns the record?

Ask what is logged: inputs, outputs, decisions, and approvals. You should be able to see why the system did what it did, and who authorized it. That record is also your evidence when a client, an auditor, or a regulator asks.

6. What will you not do?

Good partners know their boundaries. An agency that guarantees search rankings, AI citations, or legal outcomes is promising something no one controls. An agency that states clearly where its work stops has thought about the rest.

How we answer them

We built Auxeon around these questions because we asked them ourselves. Work with Auxeon starts with a free consultation. If a closer examination is warranted, the next step may be a Systems Review. Every specialist works within a published mandate, and consequential decisions remain with a named human approver. Any ongoing operation is separately scoped, with responsibility and response expectations agreed before it begins.

The standard is simple to state and demanding to meet. Hold any agency, including us, to it.

Sources

  1. "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027," Gartner, June 25, 2025.↩
  2. "FTC Announces Crackdown on Deceptive AI Claims and Schemes," Federal Trade Commission, September 25, 2024.↩
  3. Aditya Challapally et al., The GenAI Divide: State of AI in Business 2025, MIT Project NANDA, July 2025. As reported in Virtualization Review, August 19, 2025.↩
Related pageSystems Review