This guide walks you through evaluating an AI vendor’s security posture and data handling practices before you sign a contract. You’ll learn what documents to request, what red flags to look for, and how to verify that the vendor’s claims match reality. Start by assembling a cross-functional team—security, legal, procurement, and the business unit that will use the AI—so you can cover technical, contractual, and operational angles.
What You’ll Need
- A list of your own security and compliance requirements (e.g., data classification, retention periods, industry regulations)
- Access to a secure document-sharing platform for exchanging sensitive files
- A standardized vendor assessment questionnaire (or a template you can adapt)
- Legal counsel to review contracts and data processing agreements
- A technical reviewer who understands cloud architecture and API security
Step-by-Step
- Define your data sensitivity and usage scope. Write down exactly what data the AI will process (e.g., customer PII, financial records, internal code). Classify it by sensitivity and note any legal or regulatory obligations that apply. This becomes your baseline for evaluating the vendor’s controls.
- Request a completed security questionnaire. Use a recognized framework like the Cloud Security Alliance (CSA) CAIQ or your own tailored list. Ask about encryption (at rest and in transit), access controls, logging, incident response, and sub-processors. Require evidence, not just yes/no answers.
- Review their SOC 2 Type II report or equivalent. Ask for the full report, not just the certificate. Verify the report is current (within 12 months) and that the audit covered the specific services you’ll use. Look for exceptions or gaps in the controls you care about.
- Examine their data handling policies. Request their privacy policy, data retention schedule, and data deletion procedures. Confirm whether they train models on your data, and if so, whether you can opt out. Ask how they handle data subject access requests and legal requests for data.
- Check their sub-processor list. Many AI vendors rely on third-party infrastructure (e.g., cloud providers, model hosting). Ask for a complete list and review each sub-processor’s security posture. Ensure the contract requires the vendor to notify you of any changes.
- Assess their encryption and key management. Confirm that data is encrypted in transit (TLS 1.2+) and at rest (AES-256 or equivalent). Ask who holds the encryption keys—if the vendor holds them, understand what that means for your data’s confidentiality.
- Test their access controls and authentication. If you’ll have admin access, ask about SSO/SAML, MFA, and role-based access control. Verify that your team’s access can be scoped to only the data and functions you need.
- Review their incident response plan. Ask for a summary of their incident response process, including notification timelines. Confirm they will notify you within a reasonable time (e.g., 24-72 hours) if your data is involved in a breach. Get this in writing in the contract.
- Conduct a live demo or pilot with your own data. Use a small, non-sensitive dataset to test how the system handles data ingestion, processing, and deletion. Observe whether data is stored in your region or transferred elsewhere.
- Negotiate contractual protections. Work with legal to include clauses for data processing, sub-processor changes, breach notification, and your right to audit. Ensure the contract specifies your data ownership and that the vendor cannot use your data for anything beyond the agreed service.
- Perform a final risk assessment. Compile all findings into a risk register. Assign a risk rating (high/medium/low) to each gap and decide whether to accept, mitigate, or reject the vendor based on your risk appetite.
Common Mistakes
- Relying on a SOC 2 certificate alone. A certificate doesn’t tell you if the controls are effective for your specific use case. Always read the full report and look for exceptions.
- Not checking sub-processors. The AI vendor might outsource data processing to a third party with weaker security. Always review the sub-processor list and contract terms.
- Ignoring data retention settings. Some AI services retain data for model training by default. If you don’t explicitly opt out, your sensitive data may be used beyond your intended purpose.
- Assuming on-premises means secure. Even if the vendor runs its own servers, physical security and internal access controls matter. Ask about their data center certifications and employee access policies.
- Not testing data deletion. A vendor may promise deletion, but if you can’t verify it, the promise is worthless. Ask for a deletion certificate or a technical proof.
Tips
- Use a scoring matrix to compare multiple vendors objectively. Weight each criterion (e.g., encryption 20%, incident response 15%) based on your priorities.
- Ask for a copy of their data processing agreement (DPA) early in the process. If they’re unwilling to share it until after a sales call, that’s a red flag.
- Check their security page for transparency—many reputable vendors publish whitepapers, security docs, and even live status pages.
- If you’re in a regulated industry, ask if they have specific compliance certifications (e.g., HIPAA, GDPR, FedRAMP) that apply to your use case.
- Talk to their security team directly, not just the sales engineer. A dedicated security contact is a good sign.
Troubleshooting
Vendor refuses to share a SOC 2 report. Ask for a summary of the audit or a letter of attestation. If they still refuse, consider whether the risk is acceptable. For high-sensitivity data, a refusal is often a deal-breaker.
The questionnaire comes back with vague answers. Push for specifics. Ask for the policy documents themselves, and schedule a follow-up call to go through them. If the answers remain evasive, treat that as a warning sign.
You find a gap in their encryption practices. Determine if the gap affects your data. For example, if they encrypt at rest but not in some internal network, ask for a risk assessment. If they can’t fix it, consider a compensating control on your side.
You discover an unauthorized sub-processor. Check your contract—many require prior notice. If you find one, ask the vendor to remove your data from that sub-processor or to provide evidence of their security. If they won’t, escalate to legal.
The vendor’s incident response plan is thin. Ask for a tabletop exercise summary or a recent incident post-mortem (redacted). If they have no examples, suggest a joint tabletop exercise before signing.
Frequently Asked Questions
Q: Do I need a separate security review for each AI feature?
A: Yes, if the features process different types of data or use different sub-processors. A single review may not cover all functionalities.
Q: What if the vendor is a startup with no SOC 2 report?
A: That’s a risk factor, but not automatically disqualifying. Ask for a security questionnaire, a penetration test summary, and references from other enterprise customers. Consider a pilot with limited data.
Q: How do I verify their data deletion process?
A: Ask for a written procedure, and test it during a pilot. After deletion, ask for a certificate of deletion or a technical confirmation (e.g., a log entry).
Q: Should I worry about the vendor using my data to train their models?
A: Only if you don’t want that. Many vendors offer opt-out options, but you must explicitly request it. Read the contract and confirm in writing.
Q: What’s the minimum security standard I should require?
A: At a minimum: encryption in transit and at rest, MFA for admin access, a clear incident response plan, and a data processing agreement. Beyond that, align with your industry’s regulations and your own risk tolerance.
Q: Can I rely on the vendor’s security certifications instead of doing my own review?
A: Certifications are a good starting point, but they don’t cover every aspect of your specific use. Always do your own due diligence, especially for high-risk data.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- AIM and Yahoo Profiles: When Did You Last Look?
- Infinity Essence in Sailor Piece: Farming and Uses
- Apollo.io Alternatives for Sales Prospecting
- Deploy AI Agents in a Service Desk With Human Oversight
- Best Smartwatch Under ₹5,000 in India: What to Buy
- Chat History vs Saved Memory in AI Assistants