keepcalculator

06 Numbers and Powers

AI Security Controls: What to Enable and Why

Card No. 06·36332 · Updated · Standard formula, shown below

Enable layered AI security controls that protect accounts and data, restrict what AI systems can access and do, and monitor their behavior. The right settings depend on whether you are using a consumer chatbot, deploying an AI feature in an organization, or building an AI application—but strong identity, data, permission, and oversight controls are useful starting points in every case.

Start with the risks and the system boundary

AI security is not a single switch. It combines familiar cybersecurity practices with protections for risks introduced by models, prompts, training data, and tool integrations. A chatbot that only answers questions has a different risk profile from an agent that can read company files, send messages, or execute code.

First identify what the system can reach and what could happen if it is misused or compromised. Map its data sources, users, connected tools, outputs, and decision points. Consider both deliberate attacks and ordinary mistakes: a user pasting confidential material into an unapproved service, a model following malicious instructions embedded in a document, or an integration acting on an incorrect response.

Use this inventory to set controls proportionate to impact. A system that drafts public marketing text may need review and data protections; one that accesses sensitive records or triggers transactions needs much tighter access restrictions, testing, and human approval.

Protect accounts, access, and connected tools

Apply standard identity and access management before granting an AI system capabilities. Require strong authentication for administrative and sensitive accounts, and use single sign-on and multifactor authentication where available. Give users and services only the permissions needed for their tasks, and review those permissions periodically.

  • Separate roles: Distinguish ordinary users, developers, operators, and administrators. Limit who can change system instructions, deploy models, connect data sources, or alter safety settings.
  • Use least privilege: Give an AI integration read-only access when reading is all it needs. Avoid broad shared credentials; use scoped, revocable credentials and store secrets in a managed secrets system rather than prompts or source code.
  • Constrain actions: Allowlist tools and destinations, restrict access to specific records or functions, and set limits on sensitive actions. Require a person to confirm consequential actions such as deleting data, issuing payments, or sending external communications.
  • Separate environments: Keep development and testing systems away from production data and credentials unless access is explicitly justified and protected.

Do not assume that a model’s refusal behavior is an access control. Enforce permissions in the application and the underlying service, so a persuasive prompt cannot grant access the user or model was never authorized to have.

Set data and privacy controls

Decide what information may be submitted, retained, used for model improvement, or returned to users. Read the service’s current data-handling settings and terms, because options vary by product and account type. Disable optional retention or training use when it conflicts with your organization’s requirements, but do not treat that setting as a substitute for careful data handling.

  • Classify data and block or limit sensitive categories, such as credentials, personal records, confidential plans, or regulated information, where appropriate.
  • Use data-loss prevention rules, redaction, or approved gateways to detect sensitive content before it reaches an external model.
  • Minimize what is sent: provide only the fields and excerpts needed for the task, and remove identifiers when they are not necessary.
  • Set retention and deletion practices for prompts, outputs, logs, embeddings, and uploaded files. Check backups and connected systems as well as the main application.
  • Restrict access to conversation histories and diagnostic logs; they can contain sensitive data even when the model itself does not retain it.

Retrieval systems deserve particular care. A search index or vector store can expose information if its access rules do not match the original source. Preserve user-level permissions when retrieving documents, and test that users cannot obtain another person’s records by asking the model indirectly.

Harden prompts, inputs, and outputs

Prompt injection occurs when hostile or untrusted content tries to override instructions or influence a model—for example, a web page telling an assistant to reveal private information. It can appear in user prompts, retrieved files, emails, images, or other content. Treat all such material as untrusted, even when it appears relevant to the task.

Separate trusted instructions from retrieved content, label untrusted inputs, and avoid placing secrets in system prompts. These steps can reduce risk, but they cannot guarantee that a model will always distinguish instructions from data. The stronger control is to keep sensitive information and powerful actions behind independently enforced permissions.

Validate model outputs before using them. Use fixed schemas and type checks for structured responses, escape content before displaying it in a web page, and validate generated queries or commands before execution. Do not pass model-generated code, database statements, or shell commands directly to a powerful execution environment. Apply ordinary application security controls, including input validation and output encoding.

Test, monitor, and prepare to respond

Test the complete application, not just the model’s answers. Include normal use, edge cases, unauthorized access attempts, prompt injection, misleading retrieved documents, and failures in connected tools. Check whether the system leaks data, takes actions outside its intended scope, or gives users a false impression that an uncertain answer is verified.

Monitor usage and security-relevant events, while collecting only the logs needed and protecting them appropriately. Useful signals may include unusual request volume, repeated blocked actions, unexpected tool calls, access to sensitive sources, and changes to model or system configuration. Alerts should reach people who can investigate; logging without an owner or response plan has limited value.

Prepare a response process before launch. Define who can disable a model integration, revoke credentials, block a data source, preserve relevant logs, and notify affected teams. Reassess controls after changes to models, prompts, permissions, data sources, or tools. A change that seems small—such as adding web search or file access—can materially change the threat surface.

Choose controls for the deployment

For an individual using a chatbot, prioritize account security, approved services, data minimization, and checking important outputs against reliable sources. For an organization, add centralized identity, policy enforcement, retention controls, auditability, and a documented approval process for sensitive use. For a product team, build authorization, validation, rate limits, monitoring, and human review into the application architecture rather than relying on model instructions.

Compare controls by the risk they reduce, who operates them, and whether they apply to every route into the system. A provider setting may not cover data copied into another tool; an internal policy may not technically prevent a user from doing so. Prefer controls that are enforceable, testable, and assigned to a clear owner.

Common misconceptions can lead to gaps:

  • “The model is private by default.” Data handling depends on the service and configuration; verify current settings instead of assuming.
  • “A system prompt is a security boundary.” Instructions can influence behavior, but application permissions must enforce access and actions.
  • “A human in the loop makes everything safe.” Review helps only when the reviewer has time, context, and authority to reject an action.
  • “One test proves the system is secure.” Models and connected systems change, so testing and monitoring need to continue.

Frequently Asked Questions

What is the most important AI security control?
There is no single control for every deployment. Strong access management and least privilege are foundational because they limit what a compromised account or manipulated model can reach.

Should I disable AI training on my prompts?
Review the service’s current data settings and your organization’s rules. Disabling optional training use can be appropriate for sensitive work, but it does not necessarily disable storage, logging, or other processing.

Can prompt injection be prevented completely?
No single prompt technique reliably eliminates it. Treat external content as untrusted, limit tool permissions, protect sensitive data, validate outputs, and test likely attack paths.

Should AI be allowed to take actions automatically?
Only when the action is low risk, tightly scoped, and tested. For consequential or hard-to-reverse actions, require explicit human confirmation and enforce limits outside the model.

How often should AI security controls be reviewed?
Review them on a regular schedule and whenever the model, connected tools, data sources, permissions, or intended use changes. Re-test controls after significant changes.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →