The Agent That Could Not Leave Pilot
A Copilot Studio governance teardown
The framework post set out three layers that decide whether an enterprise AI initiative becomes a production capability or a long-running experiment. Discipline, governance, adoption. The first deep dive covered discipline, one lever at a time. This one is the second layer, and it is where most agents actually stop.
Not because they are bad. Because nobody made five decisions early enough.
What follows is a teardown of one agent. It is a composite, realistic in shape and drawn from patterns I have seen more than once, not tied to any one client. An internal HR policy assistant built in Copilot Studio. Leave entitlements, expense rules, the questions HR answers forty times a month. It demoed in March. In September it was still in pilot with eleven users.
The agent was fine. Every one of the five decisions below had a fast version and a shippable version, and in March the fast version won each time.
Decision 1. Which environment it is born in
What we had. Built in the default environment, because that is where the “create an agent” button lands you. Knowledge sources pointed at the open web plus a SharePoint site. No data policy applied, because the default environment is the one everybody shares and nobody owns.
What that cost. When HR asked to put it in front of four hundred staff, the answer was not “yes, next week”. It was that the agent could not be published where it lived, and moving it was an export and rebuild rather than a copy.
What we changed. Three environments, treated as governance zones with different data policies behind them. A red zone for prototyping with wide access and no expectation of survival. A yellow zone allowing internal sources plus a small list of approved external domains. A green zone for production, internal knowledge only, connectors restricted by policy. Build anywhere, ship in green.
The trade-off, honestly. Builders dislike this, and they are not wrong. Prototyping in one environment and shipping from another adds a step that a single “publish” button did not have. You are trading a small, permanent friction for a rebuild you would otherwise pay for once, at the worst possible moment, in front of a sponsor.
Quick check: if this agent had to serve the whole department tomorrow, could it be published where it currently lives?
Decision 2. Whose permissions the answers inherit
What we had. Knowledge grounded in “the HR site”. The pilot group were all HR staff, so every document the agent cited was one the reader could already open. Nothing looked wrong, because nothing was wrong yet.
What that cost. Widening the audience turns a grounded agent into an excellent oversharing detector. It does not break permissions. It surfaces, at conversational speed, the documents that were already open to everyone and had simply never been found. A restructure plan in a folder somebody forgot to scope is a Tuesday afternoon problem, not an AI problem, but the agent is what makes it visible.
What we changed. Knowledge scoped to a curated library rather than a whole site, with an owner responsible for what goes in it. Then a permissions pass on the source before the audience widened, not after.
The trade-off, honestly. Curating a library is unglamorous, ongoing, and belongs to a named human. The alternative is discovering your permissions model through a chatbot.
Quick check: take your least-privileged intended user, and ask the three most sensitive questions you can think of as them. What comes back?
Decision 3. Who can change it, and who can see how it is doing
What we had. Three people with full edit rights on the agent. Two of them needed nothing more than the usage numbers, but the numbers sat behind the same access as the instructions, so full edit is what they got.
What that cost. No separation between the people who look at an agent and the people who can change its behaviour. When an answer changed, the honest response to “who edited this” was a guess.
What we changed. Editing rights limited to two named makers. Stakeholders moved to read-only analytics access, which now exists as its own role, so visibility no longer requires configuration rights. Agent inventory pulled into a central view rather than living in whoever’s memory. Microsoft has been consolidating this into a single control plane across Copilot Studio and Microsoft 365 agents during 2026, which makes the inventory question much easier to answer than it was a year ago.
The trade-off, honestly. More roles is more administration, and in a ten-person pilot it feels like paperwork. It stops feeling like paperwork the first time someone asks why the agent’s tone changed.
Quick check: can you say today who last changed the agent’s instructions, and why?
Decision 4. How a change ships
What we had. Changes made in the live agent while users were talking to it. This is the fastest way to fix a typo and the fastest way to lose an afternoon.
What that cost. No version history worth the name, no rollback, and no way to test a change against anything before real people met it. Every edit was a small unreviewed production deployment.
What we changed. The agent lives in a solution. It moves dev to test to production through a pipeline. Direct editing in production is closed. Anything that changes behaviour, instructions, topics, knowledge sources, gets deployed rather than typed.
The trade-off, honestly. A one-word fix now costs a deployment instead of thirty seconds, and there are Fridays where that is genuinely annoying. What you buy is the ability to answer “what changed and can we put it back” without opening a support case.
Quick check: if today’s change is wrong, what is the rollback, and how long does it take?
Decision 5. What happens when it is wrong
What we had. Nothing formal. A wrong answer went into a Teams channel, got a reaction, and stayed there. No owner, no route, no cadence.
What that cost. The slowest and most expensive failure of the five. Trust does not collapse when an agent is wrong. Users expect that. It collapses when being wrong has no consequence, because that tells them nobody is minding it. Adoption, layer three, dies quietly here.
What we changed. A named owner. A feedback route that lands in the same queue as everything else HR triages. A monthly look at the questions the agent handled worst. And an explicit escape hatch in the instructions, so the agent says “this needs HR to action, please raise a ticket” instead of improvising, which is the verification lever doing governance work.
The trade-off, honestly. A review cadence is a standing commitment on somebody’s calendar forever. Every agent you deploy adds one. That is a real reason to deploy fewer, better agents rather than a portfolio of half-owned ones.
Quick check: when the agent is wrong on a Friday, who finds out, and by when?
One more line item
Forecast the credit and model usage before the audience widens, not after. Eleven pilot users and four hundred staff are different financial products, and the usage estimator tooling now covers enough of the surface to give you a number worth taking to a budget conversation. An agent that stalls on an unexpected invoice stalls just as completely as one that stalls on a policy.
What actually changed
Not a single topic was rewritten. No new capability was added. The agent that went live in September answered the same questions, the same way, as the one that demoed in March.
What changed is that it became movable, auditable, reversible and owned. That is the whole of layer two. Governance is not a feature you add to an agent. It is the small number of decisions that determine whether the agent you already built is allowed to matter.
Most of these decisions are cheap in March and expensive in September. All five of them are questions, not projects. If you are running an agent in pilot right now, the five quick checks above take about twenty minutes, and they will tell you exactly which of them you have already deferred.
Next in the prompt discipline series: Context, the second lever, and why precision almost always means less.
