AI Deployment Testing: Why Deployment Milestones Exist

ED-209 was wheeled into an OCP boardroom and asked to perform. A junior executive played the armed suspect, dropped his weapon on command and the machine shot him anyway. The demonstration was the only test it had ever faced. It was conducted in front of the people whose approval mattered most.

That sequencing problem is not fictional. AI deployment testing works only when the pass criteria are written down before the system runs, and when someone holds the authority to stop the deployment when those criteria are not met. A deployment milestone is a formal checkpoint where a system cannot progress from one stage to the next without documented evidence that it behaves as specified.

A demonstration is not a test

A test has pass criteria defined before it runs. A demonstration has an audience.

Pilots routinely conclude without anyone having agreed what “working” meant. The system produced outputs, people looked at them, the outputs seemed reasonable and the pilot was declared a success. Nobody specified the acceptable rate of incorrect outputs or the conditions for halting, so the exercise could not fail. Defining pass criteria in advance forces the difficult conversation early: how many hallucinated case references are tolerable in a legal research assistant, or what false-negative rate is acceptable for an AI triage tool in a clinical setting.

Nobody had established under controlled conditions that ED-209 could recognise compliance and stand down. The boardroom found out at the same moment the machine did.

Milestones exist because failure modes appear under real conditions

AI systems fail differently from conventional software. A payroll system either calculates correctly or it does not, and its failure modes are enumerable in advance. A language model in a customer service workflow meets inputs no test suite anticipated, and its behaviour degrades in ways that depend on context, phrasing and surrounding tooling.

The stages themselves are unremarkable:

• A controlled test environment.
• Limited internal use.
• Restricted production with human review.
• Full production.

What makes them effective is that each transition requires evidence rather than enthusiasm, and that the evidence addresses behaviour rather than function. Function is whether the system produces an output. Behaviour is what it does at the edges: when the input is adversarial, when the retrieved context is wrong, when a user asks for something outside its remit. Our AI Behaviour Verification work exists because that is where most pilots are weakest; AI safety testing concentrates on the happy path and barely touches the failure path.

Those tests only mean something when they run somewhere the system cannot reach live records, which is why test environment separation sits underneath every milestone rather than beside it.

Record at each milestone what was not tested as well as what passed. A system approved with an explicit note that its behaviour under prompt injection was never assessed leaves the gap visible to whoever inherits it.

Each milestone needs an owner with authority to stop

A checkpoint that cannot block a deployment is a status report. The pattern is recognisable: the review is scheduled, the evidence is incomplete, the business sponsor points to a launch date already communicated to customers and the system proceeds with follow-up actions nobody tracks. The process was followed and the milestone held nothing.

Ownership needs three things:

• A named individual rather than a committee. Shared accountability produces no decision at the moment one is required.
• Separation from the delivery team. The person accountable for the timeline cannot credibly be the person accountable for the halt decision.
• An escalation path with real authority. Overriding a milestone should be a decision a named executive makes and records, not a default that arrives by silence.

The failure is rarely the absence of a review process. It is that the person who could stop a launch is the person whose position depends on it proceeding.

Change management must cover model updates

Most AI change management processes govern initial release and then stop. The system passes its checkpoints, enters production and the governance record closes.

That does not survive contact with how AI systems change. The underlying model is updated by the supplier. The retrieval corpus is expanded. A system prompt is adjusted to fix a specific complaint. Each alters behaviour, and none typically triggers a review because none looks like a release. The version recorded at approval no longer describes the system in production, and the approval it supported has quietly expired.

A model version change deserves the same scrutiny as a code deployment, because the change is opaque. You cannot read a diff on a model update. The only way to understand what changed is to re-run the behavioural tests that qualified the system, which argues for a repeatable test suite rather than a one-off exercise.

Treat model updates as controlled changes:

• A defined trigger, so a supplier version bump reaches the governance record.
• A re-run of the tests that originally qualified the system.
• A documented result, held with the original approval.
• An owner permitted to roll back or restrict use where behaviour has moved.

Common questions on deployment milestones for AI

Who should own the halt decision if we do not have a dedicated AI governance function?

The most workable owner is usually the existing information security or risk lead, provided they hold explicit written authority and sit outside the delivery reporting line. Separation from the delivery timeline matters more than the job title. One named owner with a clear escalation route beats a new committee.

How many milestones should an AI deployment have?

Four is a reasonable default: controlled test, limited internal use, restricted production with human review and full production. Fewer than three tends to collapse the distinction between testing and live operation. More than five usually means the checkpoints have become administrative rather than decision points.

What if the pilot has already gone live without defined criteria?

Define the criteria now and treat the current state as awaiting review rather than as approved. Run the behavioural tests that should have run, document what fails and what was never assessed, then decide whether the system continues. That is uncomfortable but cheaper than learning the same thing through an incident.

Where to start

If your AI systems are moving toward production without documented pass criteria, a named owner and a plan for re-testing after model updates, the fix is mostly a decision about what evidence you require and who is permitted to say no. AI deployment testing is less a technical exercise than an agreement about authority.

We build that structure with organisations as part of our AI Security Programmes, covering deployment milestones, behavioural test design and the change management that keeps systems within their approved behaviour after launch. If you are approaching a production decision and are not confident the evidence supports it, contact us and we will talk through where your checkpoints need to sit.