AI Audit Evidence: What Labs Must Share

An AI lab should share with an external auditor the model's training data documentation, evaluation results, system behavior logs, and internal governance records—enough for the auditor to independently verify claims about safety, fairness, and reliability. This article explains the specific evidence categories, why each matters, and how auditors use them.

Background: Why Audits Need More Than a Summary

External audits of AI systems have grown from a niche practice into a mainstream expectation, driven by regulations like the EU AI Act and voluntary commitments from major labs. The core challenge is that AI models are complex, and their behavior can change with new data or fine-tuning. A summary report—"we tested for bias and found none"—is not enough; auditors need raw evidence to check the methodology, re-run tests, and verify that the lab's claims are accurate.

The evidence an auditor needs falls into four broad categories: data (what the model learned from), evaluation (how it performs on standard tests), behavior (what it actually does in practice), and governance (how decisions were made). Each category answers a different question, and all are necessary for a complete audit.

Training Data Documentation

Auditors need to understand the data that shaped the model, because biases and limitations often originate there. Share:

• Data sources: A list of datasets, web crawls, or licensed corpora used, with version numbers and access dates.
• Preprocessing steps: How the data was cleaned, deduplicated, filtered, or augmented—including any steps that removed or added content.
• Data composition: Statistics on language, domain, and demographic representation, where known. For example, the percentage of text from medical sources vs. social media.
• Exclusion criteria: What was deliberately left out (e.g., personal data, copyrighted material) and why.

This documentation should be detailed enough for the auditor to reconstruct the dataset or at least verify its provenance. If the data is proprietary, the lab can provide a data sheet with summary statistics instead of the raw data, but the auditor should be able to request the raw data under a non-disclosure agreement if needed.

Evaluation Results and Benchmarks

Auditors need to see how the model performs on standardized tests, but they also need the raw results, not just the headline numbers. Share:

• Benchmark scores: Results on common benchmarks like MMLU, HellaSwag, or TruthfulQA, with the exact version of the benchmark and the evaluation harness used.
• Test sets and prompts: The specific prompts used, the model's outputs, and the scoring methodology (e.g., multiple-choice accuracy, free-form grading).
• Subgroup breakdowns: Performance across different demographic groups or domains, to reveal disparities that an aggregate score hides.
• Comparison baselines: Scores from previous model versions or reference models, to show improvement or regression.

A common mistake is to only share aggregate accuracy. For example, a model might have 90% accuracy overall but only 60% on a specific dialect or technical domain. Auditors need the disaggregated data to assess real-world fairness and robustness.

System Behavior Logs

Static evaluations are not enough; auditors need to see how the model behaves in real or simulated use. Share:

• Interaction logs: Anonymized logs of user prompts and model responses, sampled across time and user types. These show emergent behavior that benchmarks miss.
• Safety test results: Red-team reports, adversarial testing outputs, and any jailbreak attempts that succeeded or failed.
• Error logs: Records of instances where the model produced harmful, biased, or incorrect outputs, and how those were handled.
• Latency and failure rates: Technical metrics like response time, timeout rates, and system crashes, which affect real-world reliability.

Logs must be carefully anonymized to protect user privacy, but the auditor should be able to see the raw text (with personal identifiers removed) to assess context and nuance. For example, a log might reveal that the model gives unsafe medical advice when asked in a specific phrasing, something a benchmark would not catch.

Governance and Documentation

Auditors also need to understand the human decisions that shaped the model. Share:

• Model cards: The official documentation that describes the model's intended use, limitations, and evaluation results.
• Development logs: Records of design choices, such as why a particular architecture or training objective was chosen.
• Ethics reviews: Documentation of any internal ethics or safety reviews, including the questions asked and the outcomes.
• Incident reports: Records of any known harms or near-misses, and the corrective actions taken.

This evidence helps the auditor assess whether the lab followed its own stated processes and whether those processes were adequate. For example, if the lab claimed to have a human oversight committee, the audit should be able to see meeting minutes or decision records.

Common Problems and How to Address Them

Several challenges arise when assembling this evidence:

• Data privacy: Logs and datasets may contain personal information. Solution: Anonymize aggressively, or provide synthetic data that preserves statistical properties.
• Proprietary concerns: Labs may fear that sharing data or model weights will leak trade secrets. Solution: Use third-party auditors under NDA, or provide feature-level statistics instead of raw weights.
• Documentation gaps: Many labs do not keep thorough records. Solution: Start with a data sheet and model card, and update them throughout development.
• Benchmark gaming: Models can be overfit to public benchmarks. Solution: Share results on private held-out test sets, or use dynamic benchmarks that change over time.

Auditors should also be wary of cherry-picked evidence. If a lab only shares results on favorable benchmarks, that is a red flag. The audit should require a complete list of all evaluations run, including those that showed poor performance.

How to Choose What to Share

Not every piece of evidence is equally important. Prioritize based on the audit's goals:

1. Define the audit scope: Is the focus on safety, fairness, security, or something else? Tailor the evidence accordingly.
2. Start with the model card: This provides a baseline of what the lab claims, and the audit verifies those claims.
3. Include raw evaluation results: Auditors should be able to re-run the evaluation or at least inspect the prompts and outputs.
4. Provide access to logs: Anonymized interaction logs are often the most revealing, so include them unless privacy is impossible.
5. Document governance: Show that decisions were made deliberately, not arbitrarily.

The key principle is verifiability: the auditor should be able to check every claim independently. If a lab says "we mitigated bias," they should provide the bias evaluation data and the mitigation logs.

Costs and Practical Considerations

Preparing audit evidence has real costs:

• Time: Documenting data and evaluations can take weeks or months, especially for large models.
• Expertise: Labs may need to hire data scientists or legal experts to prepare sensitive data.
• Infrastructure: Providing secure access to logs and datasets requires technical setup.

However, these costs are lower than the cost of a failed audit or a public scandal. Many labs now build documentation into their development pipeline, making it a routine part of the process.

Frequently Asked Questions

Do we need to share the model weights?

Not necessarily. Auditors can assess many aspects without the weights, using evaluation results and logs. However, for a deep security audit, access to weights may be necessary to check for hidden backdoors. In that case, use a secure environment and NDA.

Can we share synthetic data instead of real logs?

Yes, but with caution. Synthetic data can show statistical patterns, but it cannot reveal rare edge cases that appear in real interactions. Use synthetic data for privacy, but supplement it with a sample of real (anonymized) logs.

What if we don't have documentation from early development?

Be honest about gaps. Auditors may accept retrospective documentation, but they will note that it is less reliable. In the future, implement a documentation system from the start.

How often should we share evidence?

For ongoing audits, share evidence on a regular schedule (e.g., quarterly) and whenever there is a major model update. For one-time audits, share everything at the start.

Is it okay to redact parts of the evidence?

Yes, for privacy or security reasons, but be transparent about what you redacted and why. Auditors may push back if redactions are excessive.

What if the auditor asks for something we don't have?

Explain what you don't have and why. If it's a legitimate request, work to collect it. If it's impossible (e.g., deleted logs), document that.

Related

• AI Data Observability: Anomaly Detection Limits
• How to Level Up on Steam: Fast XP Guide
• Gene Silencing Fat: What It Means in Research

in t | Training data size (GB) | GB | 1000 | 0 | 1000000
in d | Number of training epochs | - | 3 | 1 | 1000
out total | Total compute (GB-epochs) | GB-epochs | t * d | 0