We Rolled Out AI Tools Without a Security Review. What Now?

If your organisation deployed Copilot, ChatGPT Enterprise or a handful of departmental AI tools before any formal security review took place, you are not in an unusual position. You are in the most common position. In our experience, most organisations we engage with started using AI tools before they had a governance framework, an inventory or a clear sense of what data those tools were touching. How you ended up here matters less than what the structured route forward looks like.

A retroactive AI tool security review is the answer, and it follows a predictable sequence. You do not need to stop using the tools while you do it. You do not need a six-month project. You need a baseline, a risk classification and a prioritised remediation plan. This post sets out the four steps we work through in our first engagement with organisations in exactly this position.

Step 1: Build a complete AI tool inventory

Before any assessment is possible, you need to know what AI tools are actually in use across the organisation, who is using them and what data they are processing. The case for doing this properly — and the consequences of skipping it — is set out in our shadow AI discovery guide. The practical wrinkle inside a retroactive review is that AI adoption rarely follows procurement channels, so the tools you find are not the tools on your invoices.

Marketing has a ChatGPT subscription. Finance is using a forecasting plugin inside Excel. A project team has built an internal GPT. Someone in HR is drafting policies through Claude on a personal account.

We run this as a structured two-week exercise combining four inputs: a procurement and expense review to find sanctioned subscriptions, a network and endpoint scan to identify SaaS AI tools in active use, a department-by-department questionnaire issued through line managers, and a sample of user interviews to surface the shadow AI that the first three steps miss. The shadow AI discovery tools post covers the technical signal sources in more depth. The output is a single register listing every AI tool in use, the team using it, the business purpose, the data categories it processes and whether it sits inside or outside your existing security perimeter.

In a representative engagement with a professional services firm, the procurement review might surface around a dozen sanctioned AI subscriptions while the full inventory exercise typically surfaces two to three times that number in active use. That gap between what an organisation thinks it has deployed and what is actually running is the reason the inventory step cannot be skipped.

Step 2: Classify each tool by risk

With the inventory in hand, the next step is risk classification. Not every AI tool carries the same exposure, and treating them all with equal urgency is how retroactive reviews stall. We classify each tool against four dimensions: the sensitivity of the data it processes, whether that data leaves the organisation’s tenancy, the criticality of the business decisions it informs and the regulatory regime that applies to its outputs.

A marketing team using ChatGPT to draft social copy on publicly available information is a low-risk deployment. A finance team feeding management accounts into a third-party forecasting tool is a high-risk deployment. A clinical team using an AI scribe that processes patient data is typically a high-risk deployment, with potential regulatory exposure that should be assessed against UK GDPR and the relevant NHS information governance requirements by suitably qualified advisers.

The classification produces a heat map: high, medium and low risk tools, with the reasoning documented for each. This is the artefact that lets you have a credible conversation with your board, your auditors or your regulator about where your real exposure sits. It also lets you sequence the remediation work so the highest-risk tools are addressed first rather than the easiest ones.

Step 3: Assess controls against each high-risk tool

For every tool classified as high risk, the next step is a controls assessment. The question is straightforward: for this specific tool, processing this specific data, what controls exist today and what controls should exist? We work through a fixed set of control domains for each high-risk tool: identity and access, data handling and retention, prompt and output logging, model and vendor due diligence, user training, acceptable use policy coverage and incident response readiness.

The output of this step is a gap register. For each high-risk tool, it lists which controls are in place, which are partial and which are absent. It does not recommend you switch the tool off. It tells you what needs to be true for the tool to be operated safely, and where the gaps are between current state and that target.

This is the step where most academic frameworks stop being useful. NIST AI RMF tells you the categories of control that should exist. It does not tell you whether the specific Copilot deployment in your finance team has appropriate retention controls configured. The retroactive AI tool security review is the work of going tool by tool and answering that question concretely.

Step 4: Produce a prioritised remediation plan

The final step is the one that makes the previous three useful. The inventory, the risk classification and the controls assessment together produce a long list of gaps. The remediation plan sequences them.

We sequence on two axes: risk reduction per unit of effort, and dependency. Some controls are quick wins: configuring data retention settings on a Copilot tenant takes hours and closes a significant gap. Some controls require dependency work: deploying prompt logging across the organisation requires an identity foundation that may not be in place. The plan groups remediation actions into a thirty-day, ninety-day and twelve-month horizon, with named owners and clear success criteria for each.

Crucially, the plan also identifies which tools should not have been deployed in the first place and need a managed exit. This is rare. In most engagements, the answer is that almost every tool can be operated safely with the right controls. But when the answer is that a tool needs to be replaced or withdrawn, the plan says so clearly and provides a transition path.

Key questions on retroactive AI security reviews

How long does a retroactive AI security review take?

For a mid-sized organisation, the four-step sequence runs over six to eight weeks. The inventory takes two weeks, the risk classification takes one week, the controls assessment takes two to three weeks depending on the number of high-risk tools, and the remediation plan takes one week to produce and validate with stakeholders.

Do we need to stop using AI tools during the assessment?

In our general experience, no. The assessment runs alongside existing deployments. The only circumstance in which we would typically recommend pausing a specific tool is if the controls assessment surfaces an active data exposure, in which case the recommendation would be specific to that tool rather than a blanket suspension. This reflects how we have approached past engagements rather than any form of regulatory clearance; if you suspect a tool is operating non-compliantly, take qualified legal or regulatory advice on your specific circumstances.

What if we don’t have the internal capacity to run this ourselves?

This is the most common reason organisations engage us. The four-step sequence requires a mix of governance, security and AI-specific expertise that few internal teams have assembled. Our fixed-fee AI Security Assessment delivers the work with a defined scope and a defined output, subject to scope confirmation for your environment.

How do we explain this to the board?

The risk classification heat map and the remediation plan are designed to be board-ready artefacts. They translate the technical detail into a clear picture of where exposure sits, what is being done about it, and over what timescale. Most boards respond well to seeing a structured plan rather than a list of problems.

Will this satisfy our auditors or regulators?

The documented inventory, risk classification, controls assessment and remediation plan are designed to form a defensible evidence base, demonstrating that the organisation has taken a structured, risk-based approach to AI governance. UK regulators have not prescribed a single standard methodology, so whether the output satisfies a specific auditor or regulator will depend on the regime, the assessor and the organisation’s wider control environment. We can help you map the output to the framework your auditors apply.

If your organisation has deployed AI tools without a formal security review, the AI Security Gap Analysis is the structured starting point for a retroactive AI tool security review. It is delivered as a fixed-fee engagement, subject to scope confirmation, with a defined output you can take to your board, your auditors and your remediation teams.

This article is general information about QL Security’s approach to retroactive AI tool security reviews. It is not legal, regulatory or professional advice. Organisations should take qualified advice on their specific obligations under UK GDPR, sector-specific information governance regimes and any applicable regulatory frameworks.