keepcalculator

06 Numbers and Powers

AI Agent State Safety During Context Compaction

Card No. 06·80271 · Updated · Standard formula, shown below

This guide explains how to preserve an AI agent’s state—its variables, conversation history, and task progress—when context compaction truncates or summarizes the conversation window. You’ll learn practical techniques to prevent data loss and maintain agent coherence.

What You’ll Need

  • Access to the AI agent’s configuration or prompt template
  • Ability to modify the agent’s system prompt or add a state-tracking mechanism
  • Basic understanding of JSON or key-value pairs for structured state

Step-by-Step

  1. Identify critical state variables. List every piece of information the agent must remember to function correctly: user preferences, task progress, intermediate results, and any decisions made so far.
  2. Create a dedicated state block. In the system prompt, add a clearly marked section like [STATE] that will hold all critical variables in a structured format (e.g., JSON). This makes it easy to find and preserve during compaction.
  3. Update the state block after every significant action. Instruct the agent to refresh the state block whenever it completes a step, receives new user input, or makes a decision. Use a consistent format, for example:
    {"step": 3, "total_steps": 5, "user_pref": "concise", "last_result": "data_processed"}
  4. Use a summary prompt for compaction. Before the context is compacted, run a dedicated summarization step that extracts the state block and any essential conversation facts, then places them into the new context’s state block. This can be done manually or via an automated script.
  5. Implement external state storage. For critical applications, store state outside the conversation context—in a database, file, or environment variable. The agent can read/write this storage via API calls, ensuring state survives even full context resets.
  6. Test compaction with a dry run. Simulate a compaction by feeding the agent a long conversation and observing whether the state block survives intact. Adjust the prompt instructions if any variables are lost.
  7. Add a recovery prompt. Include instructions for the agent to detect missing state (e.g., “If the state block is empty, ask the user for the last known step”) to handle unexpected failures gracefully.

Common Mistakes

  • Relying on implicit memory. Expecting the agent to remember details without explicit state tracking—this fails after compaction.
  • Storing state in the middle of the conversation. If the state block is not at the start or in a fixed location, it may be truncated.
  • Using verbose or unstructured state. Long paragraphs are harder to preserve; use compact key-value pairs.
  • Forgetting to update state after each turn. Stale state can cause the agent to repeat actions or lose progress.
  • Not testing with real compaction settings. Different systems have different truncation rules; test with your actual configuration.

Tips

  • Keep the state block as small as possible—only include variables that are absolutely necessary.
  • Use a version number in the state (e.g., "v": 1) to track format changes.
  • For long-running tasks, consider breaking them into sub-tasks, each with its own state block.
  • If using external storage, ensure the agent has clear instructions on when to read and write.
  • Regularly review the state block for redundancy and remove unused variables.

Troubleshooting

  • State block is missing after compaction. Check if the block was placed too late in the context. Move it to the very beginning of the system prompt. Also verify that the summarization step explicitly copies it.
  • State is incomplete (some variables lost). Increase the priority of the state block in the prompt, or use external storage for the most critical variables.
  • Agent ignores the state block. Reinforce the instruction with phrases like “Always refer to the state block before acting” and test with a simple scenario.
  • State block becomes corrupted or malformed. Add a validation step in the prompt: “If the state block is not valid JSON, reset it to an empty object and ask the user for clarification.”
  • Compaction still loses state despite following steps. Consider using a hybrid approach: store a condensed version of the state in the prompt and a full version externally, and have the agent reconcile them.

Frequently Asked Questions

What exactly is context compaction?

Context compaction is a process where an AI system summarizes or truncates the conversation history to fit within a token limit, potentially losing details from earlier interactions.

Can I prevent compaction from happening at all?

Usually no, but you can control what is preserved by using a state block and summarization. Some systems allow you to increase the context window, but this is not always possible.

Is external state storage always necessary?

Not always. For simple tasks, a well-structured state block in the prompt may suffice. For complex or long-running tasks, external storage is more reliable.

How often should I update the state block?

After every meaningful action or user message. The more frequently you update, the less risk of losing important progress.

What if my agent doesn’t support structured prompts?

You can still use a plain-text state section with clear labels. The key is to make it easily identifiable and parseable.

Does this work with any AI agent framework?

The principles apply broadly, but the implementation details vary. Check your framework’s documentation for context management features.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →