Not subscribed? Sign up to get it in your inbox every week.
PRESENTED BY BOLT.NEW
Fastest internal app builder for operators to MVP new ideas
You know the idea is good: the dashboard, the intake form, the workflow tool your team keeps asking for.
But it's been sitting in engineering's backlog for two quarters.
With Bolt.new, you just describe what you need and watch a real, working app come to life in your browser, ready to test before your coffee's cold.
Stop waiting on someone else's roadmap to unblock your own.
Every AI agent needs a manager
Two weeks ago I was on a panel at the AI Agent Conference in New York, sitting next to Zach Lawryk, who leads Solutions Engineering at OpenAI.
He said something that has been stuck in my head ever since.
The teams getting actual results out of AI agents treat the agents like employees. One human on the team owns the agent. The agent gets recurring performance reviews. Feedback can be blunt because the agent has no feelings to bruise.
I have been thinking about that framing over for two weeks, and it explains almost every failed AI agent rollout I have seen.
The job description problem
What happens when you hire a human without a job description? The new hire shows up, makes a best guess at the work, and the manager ends up quietly frustrated that the output is not what they wanted. Nobody articulated the actual expectation up front. The hire fails because nobody told them what success looked like.
That is exactly what happens when a team deploys an AI agent without a clear owner.
"We built a sales-research agent" sounds like a project, but who decides whether the agent is doing its job? Who reviews the output every week, and who flags it when response quality drifts down? Without a name attached to it, the agent floats. The team uses it for a while, mutters that it is fine, and quietly stops trusting the output.
The fix is putting one human's name on the agent, the way you would put a name on a quota or a P&L line.
That person owns the prompt, the eval criteria, the call on when the agent needs to be tuned, and the decision to retire it if a new agent can do the job natively. If nobody has that name on their plate, the agent does not have a manager, and managerless work decays at any company (not to mention the cost of running the agent).
A direct report that changes shape every six weeks
An agent is the strangest kind of direct report: the underlying models keeps changing under your feet (and sometimes they're massive stepwise changes).
A prompt that worked beautifully in February has a different output in May because the model was updated. Sometimes the agent gets quietly better. Sometimes it picks up a subtle regression that takes weeks to surface as a customer complaint.
If nobody is reviewing the agent on a recurring cadence, you find out about the drift the way every other team finds out about regressions: from a furious account manager forwarding you a bad output.
The teams staying ahead run a real review of each major agent every two to four weeks. They sample recent outputs, score them against the eval criteria, compare against the previous cycle, and decide whether the agent needs a prompt change, a model swap, or something more drastic. That cadence is short for a reason. The ground underneath these agents moves on a monthly cycle, and a quarterly review surfaces problems a full quarter too late.
The unfair advantage: blunt feedback
Zach's other point was the one that got the loudest reaction in the room.
Coaching a junior employee takes real emotional work. A great manager has to think about how to deliver the feedback, when to deliver it, and how to soften the framing so the person hears it without getting defensive.
Most managers spend more energy on the how to deliver the message than on the actual content. What does get through is diluted, delayed, or never delivered at all.
With an agent, all of that overhead vanishes. You can write "this output is wrong, here are five examples of what right looks like, do this from now on" and ship the change the same afternoon. The agent does not get defensive about the framing, does not need to be told it is doing well overall before hearing the criticism, and does not lie awake at 11pm wondering what it did wrong.
That is the version of feedback that actually changes behavior, and the loop runs at speed: write the new eval, run it across a sample, adjust the prompt, deploy the fix before lunch.
The feedback cycle on an agent is the version of management theory operators have been wishing for since the first 360 review was invented, and almost nobody is using it.
What this looks like on Monday
Here are four questions to run on every agent in your business. If you cannot answer them cleanly, that agent does not have a manager:
Who owns this agent?
What does good output look like, and what does bad output look like?
When was the last time someone reviewed a sample of its outputs against that bar?
What is supposed to happen the next time the underlying model is upgraded?
The cost of answering those questions is one afternoon per agent. The cost of not answering them is twelve agents quietly running in production whose output nobody on the team actually trusts.
Treat the agent like an employee. Give it an owner, a job description, a review schedule, and feedback you would never give a human.
Till next time,
Chris
PS: A bonus is costs and ROI become trackable for the first time: one name on the agent means one budget line and one set of outcomes, so you can tell which of your agents are paying for themselves.
