What Should I Check Before Deploying an AI Tool Company-Wide?

You have selected an AI tool. Procurement is close to signing and the rollout date is already on someone’s slide. Most rollouts fail on decisions taken before launch: where data travels, who and what holds access, what the supplier will commit to in writing and whether monitoring exists on day one. Written for IT directors and COOs in mid-sized UK organisations.

Reviewed by Jason Holloway.

What should I check before deploying an AI tool company-wide?

Six checks, in this order: data flows, access and identity, supplier posture, usage policy, staff training and monitoring. The first two are architectural and the last three operational. Supplier posture is contractual, so settle it before signature.

The order matters. If data flows and access are wrong, no policy corrects them afterwards. Negotiating power disappears the moment the invoice is paid. Policy, training and monitoring can improve after launch, though each should exist in draft form before it. Our AI Security Gap Analysis applies the same six domains to tools already in production, which is where organisations discover they skipped several of them.

What are the main risks of a company-wide AI rollout?

Four risks dominate a company-wide rollout.

• Data leakage through integrations, where a connector pulls in far more than the use case needs.
• Standing machine permissions that persist whether or not a human is present.
• Aggregation exposure, where output combines sources into something no single user was entitled to see.
• Evidence gaps, where the decision was sound but undocumented, so it cannot be shown to an auditor.

Policy failures are recoverable. A serious AI deployment risk assessment weighs the other three, which are architectural.

Which of these checks actually matter at our size?

All six matter. The depth varies.

Below roughly 100 staff, data flows and usage policy carry most of the weight. A two-page acceptable-use note and a documented answer to where the prompt text goes covers a lot of ground.

Between 100 and 500 staff, access and identity becomes the binding constraint. Departments buy their own tools, single sign-on coverage is partial and integrations grant AI systems standing access to mailboxes and file stores. This is the band where Shadow AI accumulates fastest and where staff adoption outside approval becomes a discovery exercise rather than a policy one.

Above 500 staff, or in any regulated setting, supplier posture and monitoring dominate. You will be asked to evidence decisions, and an internal review with no record is not evidence. Formal assessment against ISO 42001 or the EU AI Act becomes proportionate rather than premature.

What does a data flow check involve in practice?

Answer three questions and write the answers down.

What does the tool receive? Prompt text is obvious. Less obvious is everything an integration pulls in automatically: calendar contents, document bodies, chat history, CRM records.

Where is that data processed and stored, and for how long? You need a jurisdiction, a retention period and a clear statement on whether inputs train or improve models. Enterprise tiers often differ from the tier your team trialled.

What comes back out, and who can see it? Output mixing several sources can reveal something no single user was entitled to see. Every individual permission can be correct while the combined output is not.

How do we assess access and identity properly?

Separate two things that are routinely conflated: who may use the tool, and what the tool may reach.

User access is the easier half. Put the tool behind your identity provider, group-scope it and confirm that deprovisioning removes access rather than leaving an orphaned local account.

Machine access is where the risk concentrates. AI tools frequently request broad, standing permissions to mailboxes, drives and repositories. Those permissions persist regardless of whether a human is present, which makes the AI system an authority holder rather than a passive application. Scope every connector to the narrowest set of resources the use case supports, use auditable service accounts and set a review date. Our AI Behaviour Verification work exists because gap analysis findings routinely show permission grants and observed behaviour diverging.

What should we require from the supplier before signing?

Written answers, not sales assurances, on five points.

• Data residency and retention.
• Whether your inputs train their models, and how to opt out.
• The sub-processor list, and how changes are notified.
• Incident notification timelines.
• What happens to your data at contract end.

Ask too what evidence they can supply to support your own compliance position: artefacts you can hand to a regulator or auditor, not a marketing page. Suppliers answer these questions properly while the contract is unsigned. The longer form sits in how to assess an AI supplier’s security.

Can you give a worked example?

A finance and operations team adopts an AI assistant to draft supplier correspondence and summarise contracts.

Data flows: contract bodies leave the environment, so residency and retention terms are checked and a no-training commitment is obtained in writing. Access: the connector is scoped to one contracts library rather than the whole file store, and legal’s employment folder is excluded. Supplier posture: incident notification of 72 hours and a documented deletion process at contract end. Policy: no personal data of candidates or employees, no unreviewed output sent externally. Training: forty minutes for the eight people using it daily. Monitoring: monthly review of usage volume and connector scope, with any new integration treated as a fresh decision.

That is a fortnight of work, not a quarter.

What monitoring do we need after launch?

Three things, minimum. Usage visibility, so you know who is using the tool and how much. Configuration drift detection, so a widened permission does not pass unnoticed. And a reporting route staff will actually use when output looks wrong.

The third matters more than it sounds. Most AI failures inside organisations surface as someone noticing an odd answer, not as an alert. Without a low-friction way to raise that, the signal never reaches you. Quarterly review is reasonable for a single tool, monthly if the estate is growing.

Where should we start if the rollout is next month?

Data flows and machine access. Those two checks catch the failures that are expensive to reverse, and both can be completed in days. Draft the usage policy in parallel, then build training and monitoring in the first month of live use.

Related reading: AI governance. Contact us if a short conversation would help you scope the work.