Adding an AI Feature to Your Product: What Are the Security Risks?

Adding an LLM feature introduces four risk classes that traditional application security does not cover: prompt injection, data leakage through the context window, insecure output handling and model supply chain exposure. Severity depends on what the feature can read and what it can do. This page answers the questions product owners and CTOs ask before launch.

What are the security risks of adding an AI feature to our product?

Four classes cover most of it. Prompt injection means untrusted text reaching the model can override your instructions. Data leakage through the prompt means documents, records and system details placed in the context window can be extracted by a determined user. Insecure output handling covers model output rendered or executed downstream, and supply chain exposure covers the model, its hosting provider, plugins and retrieval sources as new dependencies.

Each class maps to an entry in the OWASP LLM Top 10, which is a workable structure for a first review of a new feature within our AI Security Projects scope.

What is the OWASP LLM Top 10?

The OWASP LLM Top 10 is a published list of the most significant security weaknesses in applications built on large language models. It covers prompt injection, sensitive information disclosure, insecure output handling, supply chain vulnerabilities and excessive agency, among others. It is a descriptive checklist rather than a certification standard.

What is prompt injection and why does it matter for a product feature?

Prompt injection is the LLM equivalent of injection attacks in traditional applications, with one difference: the model reads instructions and data as the same text, so they cannot be reliably separated. Direct injection is a user typing instructions that override your system prompt. Indirect injection is a document, email or support ticket your feature processes carrying hidden instructions the model obeys.

Impact scales with capability. A feature that only summarises text carries limited risk, while one that can call an API or write to a database turns injection into a route to real action.

How does data leak through the context window?

Anything placed in the prompt is potentially recoverable by the user. Three leaks are common: system prompts containing business logic or pricing rules, retrieval results pulled without applying the requesting user’s permissions so user A receives content only user B should see, and other customers’ data where conversation history or embeddings are shared across tenants.

Retrieval augmentation sharpens the last two, because permission filtering has to happen at query time rather than at display time. Context boundary testing sits alongside an AI Security Gap Analysis.

What is insecure output handling?

Insecure output handling occurs when your application trusts what the model returns. Output rendered as HTML creates a cross-site scripting route, output used in a query creates injection, output passed to a shell or code interpreter creates remote execution and output used as a URL your backend fetches creates server-side request forgery. Treat model output as untrusted user input: validate, encode and constrain it as you would a form submission.

How do we triage AI feature risk by use case?

Severity follows what the feature can read and what it can do. Read-only features over public content sit at the low end, where system prompt disclosure, inaccurate output and cost abuse are the exposures. Read-only features over private data put data leakage first, making retrieval-time permission filtering and tenant isolation the controls that matter. Features that take action raise injection to critical, because every action the model can trigger is one an attacker may trigger through crafted input.

Features executing generated code or running autonomously across several steps are the highest tier, the territory of AI Behaviour Verification.

What should be on our pre-launch security checklist?

The common failure modes cluster into a short checklist. Work through it before the feature reaches a production environment.

• Enumerate every data source the model can read and confirm permission filtering happens before retrieval.
• Enumerate every action it can take and apply least privilege.
• Place a human confirmation step before any irreversible or externally visible action.
• Encode before rendering, parameterise before querying and sandbox before executing.
• Test with adversarial input, including injection hidden inside ingested documents.
• Log prompts, retrieved context and actions so an incident can be reconstructed.
• Apply rate and spend limits.

The same sequence appears in our answer on how to test an LLM application before launch.

Does traditional application security testing cover this?

Partly. Authentication, authorisation, transport security, dependency management and infrastructure hardening all still apply. What conventional testing does not cover is a non-deterministic component: a penetration test looks for a fixed flaw in fixed code, while an LLM feature can respond differently to the same input.

The question therefore shifts from whether a vulnerability exists to how reliably the system refuses an unsafe instruction, which requires repeated adversarial testing across a range of phrasings and a tolerance decision on acceptable failure rates. A customer-facing chatbot is the clearest example of the difference.

Who owns AI feature security: product, engineering or security?

The risk sits across all three, which is why it often falls between them. Product owns what the feature is permitted to read and do, and that decision drives severity more than any technical control. Engineering owns output handling, permission filtering and action scoping. Security owns the threat model, adversarial testing and the launch acceptance criteria.

The workable arrangement is a documented decision record naming data sources, permitted actions and residual risks accepted at launch, signed off by all three. AI governance frameworks such as ISO 42001 provide an auditable structure for it.

When should we bring in external review?

Before launch rather than after, and earlier if the feature can take action, reach private customer data or execute generated code. A design-stage review costs less than remediating a launched architecture, because the decisions that matter most, which data the model sees and which actions it can trigger, are architectural and expensive to reverse.

If the feature is read-only over public content, an internal review against the checklist above may be sufficient. For related reading, see our AI Security Projects answers and our entry on Shadow AI in product teams.