Context Engineering for AI Agents: Cost and Reliability

TL;DR: Context engineering decides what an agent sees at each step: instructions, tools, retrieved evidence, history, and task state. Start by measuring that context and removing irrelevant material. Evaluate context changes on task quality and total resource use. A cached-input rate discount is not a measured reduction in total agent cost.

Key Takeaways

• Prompt engineering shapes an instruction; context engineering manages the information available throughout a workflow.
• Prefix reuse can reduce eligible input processing and billing, but cache writes, misses, output tokens, and tool costs remain.
• Stable tool definitions can help prefix reuse. Tool authorization must still be enforced by the application.
• Store large observations outside the active window with recoverable references and access controls.
• Preserve concise failure evidence and unresolved constraints when compacting history.
• Evaluate recitation, examples, retrieval, and compaction on the actual task; none has a universal accuracy or savings multiplier.

Context Engineering for AI Agents: 6 Lessons from Production Systems

What should you change first in an agent's context?

Capture one representative run and identify which inputs recur, which help solve the task, and which are carried forward unnecessarily. Use the symptom to choose a first intervention:

This is a proposed diagnostic workflow, not an experiment run for this article. The six principles below draw on Manus's production account and Anthropic's context-engineering guidance, checked September 11, 2026. Their reported experiences are not measurements of your application.

1. How do you design an agent around the KV cache?

A model's KV cache stores intermediate attention state. Cross-request prefix caching can reuse computation for matching input prefixes. Provider billing rules are a separate layer: a matching prefix must also meet that provider's model, token-length, lifetime, and configuration requirements.

Anthropic's prompt-caching documentation distinguishes base input, cache writes, cache reads, and output. Do not apply a cached-input discount to the entire bill. On self-hosted inference, vLLM's automatic prefix caching guide explicitly distinguishes reduced prefill work from output-token decoding.

Under what assumptions could a 10x figure apply?

Illustrative arithmetic, not a price quote or measured workload: suppose the selected pricing tier charges a cache read at 0.1 × the normal input rate. For the eligible input tokens that actually hit, the rate ratio is 10:1. If only 90% of those input tokens hit, and we temporarily ignore writes, the effective input-rate fraction is:

That is about 5.26x lower cost for the modeled input component, not 10x lower total session cost. Output tokens, tools, browser time, storage, and cache-write charges are excluded from this calculation.

For a separate example with a 1.25 × write rate, one write followed by nine equal-length reads costs 1.25 + 9 × 0.1 = 2.15 base-prefix units, versus ten uncached reads. That is about 4.65x lower prefix cost. It assumes the cache remains valid and all nine reads hit; it still excludes new suffix tokens and output. Verify the actual model's multipliers rather than treating these assumptions as universal pricing.

What breaks prefix reuse?

A changing timestamp near the start of a reusable prefix can prevent later tokens from matching. Reordering tool definitions, serialization, or examples can have the same effect. A stable earlier segment may still be reusable; changing one token does not necessarily invalidate an entire provider-managed cache.

Keep stable instructions and tool definitions ahead of volatile observations where the API's format allows it. Inspect the actual serialized request and provider usage fields. A stable prefix does not guarantee reuse after eviction, expiration, routing changes, or a request below the minimum cacheable length.

Compacting history may sacrifice prefix reuse to preserve task quality. Evaluate that trade-off rather than preserving an ever-growing history solely to maximize cache hits.

2. Why mask tools instead of removing them?

Manus describes maintaining stable tool definitions while constraining selection. This can help reuse a prefix and preserve the meaning of previous tool calls. It is a design option, not proof that keeping every tool always costs less than retrieving a small relevant tool set.

Anthropic's tool-definition guide documents tool_choice variants such as auto, any, tool, and none, with model and thinking-mode restrictions. These are provider-specific controls, not a universal arbitrary-subset authorization API. The older labels required and specified should not be presented as Anthropic's schema.

Apply authorization when dispatching a call regardless of what the model sees:

If a tool is unavailable, explain that boundary in current context. Compare stable schemas with tool retrieval using actual token usage and task success; neither approach is universally optimal.

3. How do you use the filesystem as context?

Keep large observations in retrievable artifacts and place only relevant excerpts in the active window. Anthropic describes structured note-taking and compaction for long-running tasks; Manus describes using the filesystem to avoid carrying every observation forward.

Conceptual artifact record:

A summary is lossy. Recovery depends on the original artifact remaining accessible and on the reference identifying the right version. A URL alone may later serve different content, and a local file path does not establish permission to read it across tenants.

Count tokens with the relevant tokenizer or provider usage; string length is not token count. For persistence boundaries, see Agent Memory.

4. How does recitation help maintain task focus?

A concise current-state note can restate the objective, completed work, constraints, and unresolved questions before the next action. Manus describes a task-plan pattern; the Lost in the Middle paper reports position-sensitive performance on the models and retrieval tasks it studied. That research does not establish one optimal insertion position or recitation interval for every current model.

Conceptual note for a code migration: “Replace the logging adapter; preserve the public API; the parser change is verified; the background worker is still unreviewed.” Keep it consistent with actual progress and remove obsolete instructions.

Compare task completion and constraint violations with and without the note, holding the model and case set fixed. Recitation adds tokens; it has no demonstrated universal 10–30% accuracy gain in this article.

5. Why should an agent keep error evidence in context?

A failed tool call explains what has already been attempted. Removing every failed step can lead the agent to repeat the same action. Preserve the useful cause and outcome without carrying an unlimited raw log into each request.

This is an illustrative record, not an actual request log. Avoid leaking credentials or sensitive payloads through error text. A model may use the evidence to choose another approach, but deterministic retry limits and recovery rules remain necessary.

6. When do few-shot examples start hurting an agent?

Examples can clarify a schema or teach a domain-specific distinction. They can also encourage imitation when the next case differs from the examples. Manus reports this pattern from its workflow experience; it is a reason to evaluate examples, not a reason to ban them.

Use a fixed, versioned example set with relevant edge cases. If you vary examples, change the experiment deliberately and record the revision rather than randomly shuffling a reusable prefix on every call. Assess structured-output validity and factual correctness separately: a perfectly formatted answer can still be wrong.

Is context engineering just prompt engineering by another name?

The practices overlap. Use clear instructions inside a context pipeline whose retrieval, state, and cost behavior can be inspected.

How do you validate a context change in production?

Freeze representative successful and failed cases, the model configuration, tools, and acceptance rubric. Change one policy at a time: stable serialization, observation excerpts, compaction, or task notes. Record input/output/cache usage, elapsed time, task success, and constraint violations, including incomplete attempts.

Do not claim success from lower input tokens alone. A smaller context that loses a required policy is a regression. Use the observability comparison for trace fields and the framework selection guide for state ownership.

FAQ

What is context engineering?

It is the design of the information an agent receives at each step: instructions, tools, retrieved evidence, history, and state. Its purpose is to preserve task quality while managing finite context and resource costs.

Does KV caching reduce every agent's cost by 10x?

No. A 10:1 cached-input rate ratio applies only under the selected pricing assumptions to input tokens that hit the cache. Cache writes, misses, new input, output, and external tools reduce the total-bill impact. The arithmetic above shows those boundaries.

Should tools always stay in the prompt?

No universal rule follows from cache reuse. Compare stable schemas and relevant-tool retrieval for the task. Enforce tool authorization in the application either way; model selection controls are not a security boundary.

Is a summary with a URL reversible compression?

Only if the full source remains retrievable in the correct version. Preserve an artifact or versioned reference when exact recovery matters, and test that the agent can find omitted details when needed.

How often should the agent restate its objective?

Choose an interval or event trigger from observed failures, then evaluate it. There is no fixed number of steps or tail-token position that this article establishes for all models. Keep notes concise and current.

Which context-engineering technique should a team start with?

Use a representative trace to identify the problem. Inspect cache usage for repeated prefixes, excerpt oversized observations, preserve error evidence, and test compaction when history becomes unmanageable. Judge each change on quality and total resource use together.

Sources

• Manus context-engineering lessons
• Anthropic effective context engineering
• Anthropic prompt-caching behavior and pricing categories
• Anthropic tool definitions and tool-choice restrictions
• vLLM automatic prefix caching and prefill scope
• Lost in the Middle research paper