AI Agents Without an LLM: When to Use Code

Use ordinary code instead of a large language model (LLM) when an agent’s task has clear rules, predictable inputs, and a finite set of outcomes. Code is usually faster, cheaper to run, easier to test, and more reliable for repeatable work; an LLM is useful when the task depends on interpreting ambiguous language or adapting to unfamiliar information.

What an AI agent does—and when it needs an LLM

An agent is software that takes in information, chooses or performs actions toward a goal, and may repeat that process until it finishes or needs help. The label does not require a particular technology. An agent can be a small program following explicit rules, a system that calls an LLM, or a combination of both.

An LLM generates or interprets language based on patterns learned during training. It can help turn a loosely phrased request into a structured task, summarize unstructured documents, or select a next step when the right response is not captured by a fixed rule. But if the work is already described by exact conditions and operations, putting an LLM in the middle adds uncertainty without necessarily adding useful capability.

For example, a program that checks whether an order total exceeds a configured threshold and routes it to a review queue needs arithmetic and a conditional—not language generation. A service that reads a customer’s free-form message and determines what kind of issue it describes may benefit from an LLM, particularly when people express the same intent in many different ways.

Tasks that are better handled with code

Code is a strong fit when requirements can be stated as a procedure and the needed data is available in a predictable format. Common examples include:

• Validating fields, checking schemas, and rejecting missing or out-of-range values.
• Sorting records, removing duplicates, applying filters, and calculating totals.
• Moving files according to fixed naming rules or generating reports from structured data.
• Calling an API with known parameters, retrying on specified errors, and recording the outcome.
• Routing a request based on explicit categories, permissions, or thresholds.
• Running scheduled tasks such as refreshing a cache or checking whether a process completed.

These operations have observable inputs and expected outputs. A test can verify that a known input produces the correct result, and a failure can be tied to a specific condition. For tasks such as financial arithmetic, access checks, or inventory updates, exact logic is especially valuable: small variations in wording should not change the result.

How a code-first agent works

A code-first agent usually consists of a control loop, explicit decision rules, and tools or services it is allowed to use. The loop can be simple: read a job, validate it, perform the permitted action, check the result, and either finish or report an error. It may run once or repeat under a defined stopping condition.

For instance, a file-processing agent could:

1. Read a queue entry containing a file identifier and requested operation.
2. Check that the identifier exists and the operation is allowed.
3. Apply the relevant transformation using a library or service.
4. Verify the output and record success, failure, or a retryable condition.

Each branch can be written and tested directly. A retry limit prevents an endless loop, while logs make it possible to see which step failed. An agent can also call an LLM for one bounded subtask, such as extracting fields from a document, then pass the result through ordinary validation before making a change.

How to choose between code and an LLM

Ask what makes the task difficult. If the difficulty is computation, data movement, or enforcing a known policy, use code. If it is interpreting varied language, reconciling context, or producing a flexible response, an LLM may help. Consider these questions:

• Are the rules stable and expressible? If you can write them as conditions, a deterministic implementation is often the simplest starting point.
• Are inputs structured? A fixed API payload or database row is easier to handle with code than an unstructured conversation or document.
• Must the result be repeatable? Code is preferable when the same valid input must produce the same result.
• What happens when the system is wrong? A consequential action needs clear validation, permissions, and human review where appropriate, regardless of whether an LLM is involved.
• Can the task be broken into parts? Use code for deterministic steps and reserve an LLM for the parts that truly require interpretation.

Do not treat “simple” as a synonym for “safe.” A short script can still make a serious mistake if its requirements are wrong, its inputs are untrusted, or its permissions are too broad. The right comparison is whether each approach can meet the task’s accuracy, auditability, and operational needs.

Reliability, costs, and safety

Code does not have the generation variability associated with LLM output: for the same environment and input, a deterministic function is designed to return the same result. It can also avoid model inference costs and reduce latency. However, total cost includes development, maintenance, infrastructure, and the effort needed to keep rules aligned with changing requirements. A brittle collection of special cases can be harder to maintain than a well-designed system that uses language understanding.

LLM-based steps introduce additional considerations. A generated answer may be incorrect or inconsistent, and model output should not automatically be treated as a validated command. Systems that use one should constrain available tools, validate outputs against an expected schema, handle refusals and failures, and require approval for sensitive actions. Code-based systems also need input validation, access controls, error handling, tests, and logging; predictable behavior is not a substitute for security review.

A practical pattern is to keep authority in explicit code. The model can suggest a category or extract candidate values, while code checks whether those values are valid and whether the requested action is permitted. When uncertainty is high, missing information matters, or the action is difficult to reverse, the system can stop and ask a person rather than guessing.

Common misconceptions and failure points

One misconception is that every “agent” must use an LLM. It does not: the word describes a role or architecture, not a requirement for generative AI. Another is that code is automatically more accurate. It follows the rules it was given, including incorrect rules, and may fail when input formats change or an edge case was overlooked.

Another common mistake is using an LLM as a substitute for a rule engine. If a policy says a field must be present or a value must fall within a defined range, verify that condition directly in code. Asking a model to remember or infer an exact constraint makes enforcement harder to test.

Conversely, forcing every unclear request into rigid rules can create a maze of exceptions. If people use varied wording and the desired outcome depends on context, a model may reduce manual rule-writing—but its output still needs evaluation. Test representative cases, including ambiguous inputs, malformed data, timeouts, and unexpected tool responses. Measure whether the whole workflow succeeds, not only whether a model’s text sounds plausible.

Frequently Asked Questions

Can an AI agent work without an LLM?
Yes. An agent can use ordinary code, rules, and APIs to observe inputs, choose among defined actions, and report results. An LLM is optional, not part of the definition.

When should I use code instead of an LLM?
Use code for stable, explicit procedures such as validation, calculations, routing, and repeatable data transformations. Consider an LLM when a task requires interpreting flexible language or other unstructured information.

Are code-based agents more reliable?
They are more predictable for well-specified tasks because their behavior can be made deterministic and tested. They are not inherently correct: flawed logic, changing inputs, and unhandled failures can still cause errors.

Can an agent combine code and an LLM?
Yes. A common design uses an LLM for interpretation and code for validation, permissions, calculations, and execution. This keeps flexible reasoning separate from steps that require exact checks.

Does avoiding an LLM always make an agent cheaper?
It can reduce inference usage and related latency, but it does not guarantee a lower total cost. Engineering time, maintenance, infrastructure, and the cost of handling exceptions also matter.

Related

• Fisch Nate’s Crown: Glasses & Lantern Spots
• Remote MCP Server Checks: Which Probe Proves Usable?
• Tanuki Pons: Summer Guide, Exploration & Shrine Upgrades