keepcalculator

06 Numbers and Powers

AI-Generated Code: What to Check Before Approval

Card No. 06·29339 · Updated · Standard formula, shown below

Before approving AI-generated code, review it to the same standard as any other contribution: verify that it meets the requirements, behaves correctly, handles errors, and does not introduce security, privacy, licensing, or maintenance risks. Treat the tool’s output as an untrusted draft—not as evidence that the code is correct—and require a human reviewer who understands the change to take responsibility for it.

Why AI-generated code needs ordinary engineering review

Code-generation tools predict or assemble likely code from a prompt and surrounding context. They can speed up routine work, suggest unfamiliar APIs, or produce a first draft, but plausibility is not proof. A snippet can compile while implementing the wrong behavior, omitting an important case, or making assumptions that are not true in the application.

The same review principles apply whether a contribution was written by a person, suggested by a tool, or produced collaboratively. The difference is that reviewers may not know which assumptions the tool made, what context it lacked, or whether a confident-looking explanation accurately describes the code. Review the actual diff and its evidence, not the tool’s description of its answer.

Start with scope, requirements, and the diff

First establish what the change is meant to do and whether it does only that. Compare the implementation with the issue, acceptance criteria, design notes, and relevant existing behavior. A concise prompt or generated comment is not a substitute for a testable requirement.

  • Check the diff: Look for unrelated edits, generated files, dependency changes, broad formatting, or altered configuration that were not needed for the task.
  • Trace the behavior: Follow inputs through the code to outputs and side effects. Confirm the right function is called and that data is transformed in the expected order.
  • Check boundaries: Consider empty, missing, malformed, unusually large, and unexpected values where the requirements make them relevant.
  • Confirm compatibility: Check the project’s supported language, framework, platform, and API versions rather than assuming a suggested feature exists everywhere.

For example, a generated search filter might work for a normal query but mishandle an empty string, escape user input incorrectly, or change how existing callers interpret a missing value. Those are review questions tied to the product’s actual contract, not to whether the code looks idiomatic.

Verify behavior with tests and independent evidence

Run the project’s relevant tests, linters, type checks, and build steps. A green result is useful evidence, but it is not a complete review: tests may not cover the new path, and a test can encode the same mistaken assumption as the implementation.

Look for tests that demonstrate the intended behavior, including failure cases and boundary conditions. If the generated change modifies an existing feature, confirm that regression tests cover the old behavior that must remain. For a small change, a reviewer may also manually trace a representative input; for a complex change, inspect the test design and execution results rather than relying on a summary.

When the code uses a library or API, check authoritative project documentation or the dependency’s source as appropriate. Generated code can invent plausible method names, mix APIs from different versions, or use a real function with the wrong semantics. Confirm dependencies are available, maintained according to the project’s policy, and actually necessary.

Review security, privacy, and data handling

Security review should focus on what the code accepts, trusts, stores, sends, and exposes. AI assistance does not make an input safe or remove the need for the project’s threat model. Pay particular attention when a change touches authentication, authorization, file access, network requests, database queries, secrets, or personal data.

  • Check that access controls are enforced on the server-side or other trusted boundary, not only in the interface.
  • Look for unsafe handling of untrusted input, including injection risks, path manipulation, and improper output encoding.
  • Confirm errors and logs do not expose credentials, tokens, personal information, or sensitive internal details.
  • Review data collection, retention, and transmission against the application’s existing privacy requirements.
  • Inspect new dependencies and configuration for unexpected permissions, network behavior, or secret values.

For example, a generated database query may appear to filter correctly yet construct part of the query from raw user input. Review how values are parameterized and how authorization is applied; do not infer safety from a passing test or a comment claiming the input is validated.

Check licensing, provenance, and tool policy

Generated output can resemble code found in training or reference material, and a reviewer may not be able to establish its origin from the code alone. Follow the organization’s rules for approved tools, disclosure, and handling of generated material. Check whether the tool or workflow has restrictions on submitting proprietary code, customer data, or secrets as prompts; do not assume that a tool’s availability makes every use appropriate.

Review dependency licenses and attribution requirements through the project’s normal process, especially when the change introduces a package or a substantial copied-looking block. If provenance or a licensing concern cannot be resolved, pause approval and ask the appropriate legal, security, or open-source program contact. Avoid making unsupported claims that a piece of generated code is—or is not—copied from a particular source.

Assess maintainability and decide who owns the change

Approved code becomes part of the system that the team must debug and maintain. Check whether names, structure, comments, and error handling fit local conventions. Remove unnecessary abstraction and duplicated logic. A clever implementation that the team cannot explain may cost more to maintain than the time it saved during drafting.

Ask the contributor to explain important design choices, assumptions, and failure behavior in their own words. This is not a test of whether a person typed every line; it is a way to establish that someone can support the change. The reviewer should understand the changed paths well enough to identify likely consequences and know what evidence supports approval.

Teams can make this repeatable with a pull-request checklist: requirements met, diff scoped, tests reviewed, security and data flow considered, dependencies checked, and ownership clear. Apply deeper review according to impact and risk. A wording change and a change to authorization logic do not require identical scrutiny.

Frequently Asked Questions

Should AI-generated code get a separate approval process?
Not necessarily. It should meet the same engineering and security standards as other code. Teams may add disclosure or tool-use checks where their policies require them, but an “AI-generated” label is not a substitute for reviewing the change.

Can I approve code if all the tests pass?
Passing tests is one piece of evidence, not proof of correctness. Check whether the tests cover the requirements and important failure cases, and review the implementation for risks the test suite does not exercise.

How can I tell whether generated code is secure?
There is no reliable visual marker. Examine trust boundaries, input validation and encoding, permissions, data flows, secrets, dependencies, and error handling; use the team’s normal security review and analysis tools where appropriate.

Do developers need to disclose AI assistance?
That depends on the organization’s policy, contract, and project workflow. Teams should state expectations clearly, especially for regulated, confidential, or externally contributed work, rather than assuming one disclosure rule applies everywhere.

What should happen if the reviewer cannot explain the code?
Do not approve it yet. Ask for a simpler implementation, clearer documentation, or an explanation and evidence sufficient for someone on the team to maintain it. If a risk remains unresolved, escalate or reject the change.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →