Field NotesBest practice5 min read

Pick a first AI workflow your team won't have to babysit

New BCG research shows heavy AI oversight wears people out. Use it to choose a first workflow that saves time instead of creating review work.

Adam Blazer

Fractional Chief Intelligence Officer

A printed contract on a desk with a pen resting across it.
Photo: Blogtrepreneur · CC BY 2.0 · Wikimedia Commons

Last week, Boston Consulting Group put a name to something a lot of managers have noticed. In a survey of 1,488 full-time US workers, written up by Fortune, BCG researchers described “AI brain fry”: the mental fog people report after long stretches of supervising AI output.

The finding that matters most for operators is about which work causes it. Jobs that required heavy oversight of AI, like reading and interpreting text a language model had generated, came with 14 percent more mental effort, 12 percent more mental fatigue and 19 percent more information overload. The article contrasts that with an AI agent completing administrative tasks, which needed much less oversight.

That is a useful guide for the first decision every company has to make: which workflow do you hand to AI first?

DataExtra strain on workers in high-oversight AI jobs
  • Information overload19%
  • Mental effort14%
  • Mental fatigue12%
Compared with workers whose AI use needed less oversight, per BCG's survey. Source: fortune.com

Most companies pick the wrong first project

The first AI project most companies reach for is a writing assistant. Draft the proposals, summarize the meetings, write the marketing emails. It is easy to demo and everyone can picture it.

It is also the high-oversight work the BCG study describes. Every draft has to be read closely, checked for facts, adjusted for tone and rewritten in places. The person doing that checking is doing a different job, often a harder one. If the draft was mostly right, you have traded writing for editing, and editing someone else’s confident prose is tiring.

There is nothing wrong with writing help. It just makes a poor first project, because the time savings are hard to see and the strain on the reviewer is easy to feel. A first project needs to produce a clear win that people want more of.

Deloitte’s survey from January suggests most companies haven’t gotten there. Only 30 percent of organizations said they were redesigning key processes around AI, while 37 percent were using it at a surface level. A writing assistant handed to everyone is the definition of surface level.

Six tests for a good first workflow

When I look at a list of candidate workflows, I run each one through the same tests. A good first workflow passes most of them.

ChecklistDoes this workflow make a good first project?
  • It happens many times a week
  • The rules can be written down
  • A right answer is easy to check
  • One manager owns it
  • Mistakes can be undone
  • The data it needs is reachable

It happens many times a week. Volume is where hours come from. A task done twice a month will never pay back the setup, however painful it is.

The rules can be written down. If your best person can explain how they do it in a page, an agent can follow that page. If the answer is “it depends, I just know,” leave it for later.

A right answer is easy to check. This is the test the BCG research sharpens. Matching an invoice to a purchase order is right or wrong. Routing a maintenance request to the correct vendor is right or wrong. A reviewer can confirm those in seconds. A drafted client letter can’t be checked in seconds.

One manager owns it. Someone has to care whether it works, and that person needs authority over how the team does the task.

A mistake can be caught and undone. Pick work where an error gets caught before it reaches a customer or a bank account, or where it is cheap to reverse. Keep payments, legal commitments and anything regulatory for later, once you trust the setup.

The data it needs is reachable. If the agent needs information that lives in someone’s head or a paper file, you have a data project before you have an AI project.

A worked example: maintenance requests beat owner reports

Say you run a 40-person property management company. Your operations lead brings you three candidates.

The first is drafting monthly reports for property owners. It is high value and everyone hates doing it. But each report mixes numbers with judgment about what to tell the owner, and the reviewer has to read every line. It fails the easy-to-check test.

The second is triaging tenant maintenance requests. Hundreds come in each month by email, text and a web form. Each one needs to be categorized, matched to the right unit and sent to the right vendor, with emergencies flagged. The rules already exist in a binder. A coordinator can glance at the agent’s choice and approve it. It passes every test.

The third is reviewing new leases for unusual clauses. It is valuable, but a missed clause is expensive and a reviewer would need to read the whole lease to trust the result. Not first.

Before and after

Owner reports first

  • Reviewer reads every line
  • Errors are judgment calls
  • Savings are hard to count

Maintenance triage first

  • Reviewer approves at a glance
  • Errors are obvious and fixable
  • Savings are counted per request
Both are worth doing eventually, but only one makes a strong first project.

Maintenance triage goes first. The owner reports can come third or fourth, once the team has seen what an agent does well and has built some trust in checking its work.

Keep the tool count small

The BCG study had one more finding worth acting on. According to Fortune, self-reported productivity went up for people using three or fewer AI tools and fell sharply for people using four or more. The researchers also found that brain fry decreased when managers provided training and support on the tools.

For a first project, that argues for one tool doing one job, with someone who knows the tool sitting next to the team that uses it. Resist the urge to trial three vendors in parallel across three departments. You will learn less and wear people out faster.

It also argues for building the review step on purpose. If the agent’s output lands in the same inbox as everything else, the reviewer has one more stream to watch. If it lands in a single queue where each item shows what the agent decided and why, approving it takes seconds. In the maintenance example, that means the coordinator sees the request, the category the agent chose and the vendor it picked, with one button to approve and one to change. Design that screen before you design anything else.

Finally, plan what happens to the time you free up. If the coordinator who used to sort requests all morning now approves them in twenty minutes, decide in advance what the rest of that morning is for. Otherwise the saved time disappears into email and nobody can point to it a month later.

What to do this week

  1. Ask each department head to name the two tasks their team does most often that follow written or unwritten rules.
  2. Run each candidate through the six tests above. Cross off anything that fails “a right answer is easy to check.”
  3. For the two or three that remain, count how many times the task happened last month.
  4. Pick the one with the highest count and a manager who wants it.
  5. Ask that manager to write the rules for the task on one page. If they can’t, move to the next candidate.

The page of rules from step five is the most useful thing you will produce this month. It becomes the agent’s instructions, the reviewer’s checklist and the baseline for measuring what changed.

Sources

  • strategy
  • operations
  • agents
Adam Blazer

Adam Blazer

Adam has spent twenty years building companies where media meets technology. He has built thirteen companies, raised more than $134 million, co-founded the cable network JLTV, founded the Emmy-nominated studio Livit and runs VLO Studios. He works as a fractional Chief Intelligence Officer for companies deciding where AI should do their work.

Work with Adam

Put AI to work in one department of your company in 14 days.

If it isn’t saving the hours we agree on within 30 days, you get your money back.

Keep reading