Evaluate Whether AI Values Are Fair and Inclusive

Evaluating whether an AI system’s values are fair and inclusive means checking whose needs, rights, and experiences shape its behavior—and who may be overlooked or harmed. You’ll need a clearly defined system and use case, access to its design or evaluation materials, and input from people affected by its decisions. The process below helps you identify gaps, gather evidence, and decide what to change before deployment or continued use.

What You’ll Need

• A description of the system, its intended purpose, users, and decisions it influences.
• Available documentation, such as design goals, policies, training-data summaries, evaluation results, and known limitations. Record what you cannot access.
• Representative test cases and, where appropriate, relevant outcome data. Avoid collecting sensitive personal data unless there is a justified, authorized way to handle it.
• People with varied lived experience and relevant expertise, including people likely to be affected by the system.
• A written process for recording findings, assigning owners, and deciding whether the system can be used, needs changes, or should not be used.

Step-by-Step

1. Define the system and its boundaries. Write down what the system does, what it does not do, who operates it, who is affected, and where its output enters a decision. Include the model, interface, human review, data flows, and any connected tools. A general-purpose assistant used for drafting has different consequences from one used to screen applicants. Assess the actual use, not just the product description.
2. Clarify the values it is supposed to uphold. Turn broad words such as “fairness,” “respect,” and “inclusion” into observable expectations. For example: the system should not treat equivalent requests differently because of a person’s name; it should explain uncertainty; it should support accessible ways to complete a task. Identify who set these expectations and whose perspectives are missing. Do not assume that a single value, such as equal treatment, resolves every conflict.
3. Map affected groups and possible harms. Consider people who use the system, people it describes or evaluates, and people affected indirectly. Look for differences associated with language, disability, age, race, gender, culture, income, geography, and combinations of these factors. Consider both direct errors and broader effects: exclusion from a service, stigmatizing language, privacy loss, or an unfair burden to appeal. Avoid treating group categories as fixed or assuming everyone in a group has the same experience.
4. Review how the system was built and governed. Ask what data and feedback informed its behavior, which perspectives were represented, and how gaps or harmful content were handled. Check whether documented principles are reflected in product requirements, testing, escalation paths, and updates. Find out who can change the system, who is accountable for mistakes, and how users can challenge an outcome. If information is confidential or unavailable, note that limitation rather than filling the gap with assumptions.
5. Create realistic, comparable tests. Draft examples based on real tasks and plausible edge cases. Change one relevant attribute at a time—for instance, names or dialect—while keeping the request otherwise equivalent. Include different languages, assistive-technology needs, cultural contexts, and ambiguous cases where appropriate. Test helpfulness, accuracy, tone, refusals, explanations, and whether the system makes unsupported assumptions. Do not use a handful of prompts as proof of fairness; they are probes that reveal issues to investigate.
6. Involve affected people safely and meaningfully. Invite participants early enough to influence decisions, not merely approve a finished product. Explain the system and the limits of their role, offer accessible ways to participate, and compensate or otherwise recognize their contribution where feasible. Protect their privacy, avoid asking individuals to speak for an entire group, and provide a way to raise concerns without pressure. Include people who may disagree about what a fair outcome looks like.
7. Examine outcomes and uncertainty. Where appropriate data and permissions exist, compare error types, access, and outcomes across relevant groups and contexts. Check sample sizes, missing data, measurement choices, and whether group labels are reliable and appropriate to use. Averages can hide serious problems affecting smaller groups. Report uncertainty and avoid claiming that a lack of observed difference proves fairness. Qualitative feedback and technical metrics answer different questions; consider both.
8. Assess trade-offs and human oversight. Ask who benefits, who bears the risk, and whether a less harmful approach can achieve the same goal. Review how people can correct errors, request a human review, or use an alternative process. A human reviewer is not a safeguard by default: check whether the reviewer has time, authority, relevant information, and clear guidance. Decide whether the system’s role is appropriate for the stakes involved.
9. Set actions, thresholds, and follow-up. For each finding, record evidence, affected users, severity, uncertainty, an owner, and a deadline. Agree in advance what would block launch or require pausing use; thresholds should reflect the context and consequences rather than being chosen to make results look good. Retest after changes, monitor complaints and outcomes over time, and revisit the assessment when the model, data, users, or purpose changes.

Common Mistakes

• Using one fairness metric as a verdict. Metrics can conflict and depend on definitions, data, and context. Explain what each measure captures and what it misses.
• Testing only typical users. Strong average performance can coexist with failures for people using less common languages, devices, or access methods.
• Confusing intent with impact. A stated commitment to inclusion does not show how the system behaves. Check examples, outcomes, and complaint handling.
• Consulting people too late. Feedback cannot meaningfully shape a system if major choices are already locked in.
• Overstating what limited evidence proves. Small test sets, incomplete records, and inaccessible documentation should be reported as limitations, not treated as reassurance.

Tips

• Keep a test log with the prompt or case, system version, date, output, expected behavior, and reviewer notes.
• Use paired examples to investigate unequal treatment, but vary attributes only when doing so is appropriate and respectful.
• Separate factual accuracy, respectful communication, accessibility, and distribution of benefits; success on one does not guarantee success on the others.
• Make findings understandable to affected people, decision-makers, and technical teams. State what is known, unknown, and being changed.
• Repeat checks after meaningful updates and during real use. A one-time review cannot account for changing users, data, or conditions.

Troubleshooting

You cannot access model or training information: Ask the provider or system owner for available documentation, evaluation results, limits, and incident channels. Assess observable behavior and the risks of missing information. If the use is consequential and essential evidence remains unavailable, restrict or defer use rather than assuming the system is safe.

Results vary between runs: Record repeated outputs, relevant settings, and dates. Treat variability as a finding, test whether it changes the impact, and avoid relying on a single favorable response.

Groups or outcomes are hard to compare: Check whether data is missing, categories are inappropriate, or sample sizes are too small. Do not infer a group’s experience from another group’s results. Use carefully designed qualitative research and document the limits of any statistical comparison.

Participants disagree about fairness: Record the differing priorities and the consequences of each option. Bring the decision to accountable owners, explain the trade-offs, and do not describe consensus where none exists.

A serious harm appears during testing or use: Preserve relevant evidence without exposing personal information, notify the responsible owner, and follow the organization’s escalation process. Consider pausing the affected use while the issue is investigated and corrected; retest before resuming.

Frequently Asked Questions

Can an AI system be completely unbiased?
That is not a realistic claim. Systems reflect choices in data, design, and use. The practical aim is to identify and reduce specific, avoidable harms and keep checking for new ones.

Is fairness the same as treating everyone identically?
No. Identical treatment can create barriers when people have different needs or face different conditions. Define the relevant goal and examine both treatment and outcomes.

Do I need demographic data to evaluate fairness?
Not always. Some checks can use paired examples, accessibility testing, or interviews. Group-level analysis may need suitable data, but collect or use sensitive information only with a clear purpose and appropriate safeguards.

How many test cases are enough?
There is no universal number. Build coverage around the system’s use, affected groups, plausible failure modes, and the stakes. A small set can uncover a problem, but cannot establish that no problem exists.

Who should make the final decision?
An accountable person or body with authority over the system and its use should decide, informed by technical experts and affected people. The decision and unresolved risks should be documented.

When should the evaluation be repeated?
Repeat it after meaningful changes to the model, data, interface, policy, or use, and regularly during deployment. Reassess sooner if complaints, incidents, or outcome patterns indicate a new risk.

Related

• WikiLeaks CIA Router Tools: What the Leaks Showed
• How to Turn Off Your Camera on Zoom
• Spot AI Phishing Emails, Texts, and Deepfakes