AI Coding Agents: What They Can and Cannot Do

AI coding agents can reliably handle bounded, verifiable work in an existing codebase—such as tracing a function, making a small change, and running relevant tests—when they have suitable repository access and clear instructions. They cannot reliably infer every unstated requirement, prove a change safe just because tests pass, or take responsibility for architecture, security, and production decisions without human review.

What an AI coding agent does

A coding agent is a software tool that uses an AI model to work through programming tasks, often by reading files, proposing or editing code, and invoking tools such as a terminal, test runner, or search utility. Unlike a chat assistant that only suggests snippets, an agent may perform a sequence of actions in a repository and revise its work based on command output.

The term covers different capabilities. Some agents can only suggest patches; others can inspect a working tree, modify files, and run commands. Their actual reach depends on tool permissions, repository configuration, model capability, and any limits set by the person operating them. Access to a terminal does not mean an agent understands every command’s effects.

In an existing codebase, context matters. A request is not just a description of the desired feature: it interacts with established conventions, dependencies, tests, deployment assumptions, and code that may not be documented. Agents can discover some of that context by searching files and examining usage, but discovery can be incomplete.

Tasks they tend to handle well

Agents are most useful when a task has a narrow scope, relevant examples already exist, and success can be checked. They can save time on investigation and routine implementation, particularly when a developer can review a concise patch.

• Repository orientation: Locate a symbol, follow straightforward call paths, summarize a module, or identify files likely involved in a change. Treat this as a starting map, not a guaranteed complete account.
• Localized changes: Add a field to a familiar data structure, update a similar endpoint, adjust a small component, or apply a consistent change across a few files.
• Tests and maintenance: Draft tests for explicit behavior, update tests after an intended change, fix simple lint or formatting failures, and explain an error message using nearby code.
• Mechanical transformations: Rename or reorganize code where references are discoverable, convert repeated patterns, or update documentation to match verified behavior.
• Exploration: Compare possible implementation locations, list assumptions, and propose a patch for a human to assess.

These are tendencies, not guarantees. A seemingly small edit can affect a public interface, generated files, stored data, or a downstream service. The agent’s confidence or fluent explanation is not evidence that it found every dependency.

How reliability depends on context

An agent works from the information it can access: the prompt, files made available to it, tool results, and any other configured context. If the repository is large, the relevant rules are scattered, or a behavior depends on runtime state, it may miss important facts. Naming a nearby example, specifying constraints, and pointing to tests can help, but cannot ensure correctness.

Verification also has limits. A passing test establishes that the tested cases passed in that environment; it does not establish that all intended behavior is correct. Tests may be absent, stale, overly narrow, or themselves changed incorrectly. A command that succeeds may not exercise the production configuration. Conversely, a failure may be unrelated to the patch.

Reliability improves when the task is specific and feedback is meaningful. For example, “add input validation for this endpoint, preserve its response format, and add tests for missing and malformed values” is easier to evaluate than “make the API robust.” The agent still needs a human to confirm what “robust” means for the product and its callers.

What they cannot reliably decide or guarantee

Agents do not automatically know the product’s unwritten requirements, which trade-offs maintainers prefer, or whether an apparent inconsistency is intentional. A codebase may contain legacy behavior relied on by clients even when that behavior looks like a bug. Changing it without checking compatibility can cause a regression.

• They cannot prove correctness from plausibility. A coherent patch may have edge-case defects, race conditions, incorrect assumptions, or mismatched error handling.
• They cannot guarantee security. They may identify familiar patterns, but a review must consider the threat model, data flows, permissions, dependencies, and deployment environment.
• They cannot replace system-level judgment. Choosing service boundaries, migration strategies, compatibility policy, or operational trade-offs requires context beyond the edited files.
• They cannot safely assume every command is harmless. Tools can modify files, install dependencies, run scripts, or interact with external services. Commands and side effects need appropriate controls.
• They cannot guarantee that generated or hidden behavior is covered. Build pipelines, configuration, generated code, and external integrations may not be visible in a local test run.

A common misconception is that an agent that can edit and test code is equivalent to an autonomous maintainer. Tool use makes it more capable of acting; it does not give it organizational authority, complete context, or a dependable understanding of consequences.

A practical review workflow

Use an agent as a contributor whose work must pass the same checks as a human patch. A careful workflow keeps the task bounded and makes changes easy to inspect.

1. Define the outcome: State expected behavior, constraints, relevant interfaces, and what must not change. Include a concrete example or acceptance criteria when possible.
2. Ask for investigation before broad edits: Have the agent identify relevant files and assumptions. Correct misunderstandings before it changes multiple parts of the system.
3. Limit the scope: Prefer a small patch. Keep unrelated cleanup separate so the purpose and impact of each change remain clear.
4. Inspect the diff: Check every changed file, including tests, configuration, dependency manifests, and generated output. Look for unexplained edits or weakened assertions.
5. Run independent checks: Use the project’s documented tests, formatting, type checks, or build steps as appropriate. Review failures rather than treating a single green command as proof.
6. Review behavior and risk: Consider edge cases, backward compatibility, data handling, permissions, and rollback needs. Use specialist review where the change warrants it.

For tool access, apply least privilege: provide only the files and commands needed, and require confirmation for consequential operations. Keep secrets out of prompts and logs. Review external instructions found in repository files or command output rather than treating them as automatically trustworthy.

Choosing tasks and setting expectations

A good first task has a clear boundary, a nearby pattern to follow, and a practical way to verify the result. Examples include adding a test for a specified case, correcting a typo in a well-understood message, or updating a small function with a documented expected output. A vague request to “clean up the code” invites unnecessary changes and makes success hard to judge.

Raise the review bar as the consequences rise. Changes involving authentication, authorization, cryptography, personal data, financial records, database migrations, concurrency, or deployment deserve careful human ownership and relevant expert review. An agent may help explain code or draft a proposal, but should not be the final approver for high-impact decisions.

There is no universal accuracy rate that predicts whether an agent will succeed on a particular repository task. Results vary with the model, tools, task design, codebase, and verification available. Evaluate it on representative low-risk work, track how often its patches need correction, and decide where it fits based on observed outcomes—not demonstrations or confident prose.

Frequently Asked Questions

Can an AI coding agent understand an entire codebase?
It can inspect and summarize accessible files, but it may not hold every relevant dependency or convention in view. Treat its explanation as a navigational aid and verify important claims against the code and tests.

Can I merge code if the agent’s tests pass?
Passing tests is useful evidence, not a guarantee. Review the diff, check whether the tests cover the requirement, and run the project’s appropriate checks before deciding whether a patch is ready.

Are coding agents safe to use on a private repository?
That depends on the tool’s data handling, access controls, configuration, and your organization’s rules. Check those conditions first, avoid exposing secrets, and limit permissions and command execution to what the task needs.

Should an agent make changes across many files at once?
It can, but broad changes are harder to inspect and may compound a mistaken assumption. Break work into reviewable steps, especially when interfaces or persistent data are involved.

What is the best task to give a coding agent first?
Choose a low-risk, clearly specified task with an existing example and a testable result. That reveals whether the agent can follow the repository’s conventions before you trust it with more consequential work.

Related

• Write Shell Scripts in AWK? What AWK Can and Cannot Do
• Google Ads API Release Notes and Deprecation Deadlines
• Fiber Internet Not Available? Your Best Alternatives