Context engineering in the enterprise: what belongs in the context window?

Short answer

Context engineering is the selection and maintenance of everything entering an inference: instructions, history, tools, retrieved evidence, memory and metadata. The goal is not to fill the window with the most text, but to provide information that is relevant, current, authorized and understandable when the agent acts.

For an enterprise, the method is to map sources, define priority rules, remove noise, measure outcomes and retain a record of the context actually used. For some teams, a short and stable context is enough. For others, a multi-step agent needs adaptive context with summaries, tool boundaries and provenance controls.

From prompt to complete context

Anthropic describes context engineering as the natural progression of prompt engineering: the system must manage all available tokens, not just the instruction message (Effective context engineering). Its documentation recommends explicit instructions, structured examples and XML tags to separate instructions, data and output (Prompting best practices).

Context also includes tools and their results. A long description, stale history or unfiltered document can hide the important rule. The design question becomes: what information enables the next decision, and what must remain out of reach?

The selection flow

Nexxom diagram: start from a goal, filter sources and retain evidence of the active context.

1. Define the goal. Describe the expected decision and acceptable errors.
2. List candidate sources. Instructions, profile, memory, documents, tools and previous results.
3. Filter. Apply access rights, freshness, relevance, size and redundancy rules.
4. Structure. Separate rules, data, examples, uncertainty and permitted actions.
5. Limit. Reserve space for the answer and tool calls.
6. Trace. Record identifiers, versions, selection scores and removed items.
7. Evaluate. Compare quality, latency, cost, security and escalation rate.

Relevance before quantity

More context can mean more information, but also more contradictions. Test an incremental strategy: instruction alone, instruction plus business context, then filtered context. Measure success on a stable set and inspect errors introduced by irrelevant documents.

The Plotly matrix below uses Nexxom heuristics, not model scores.

Nexxom Plotly matrix: a 0 to 5 heuristic for review structure, not a benchmark.

Manage history and memory

In a multi-step agent, history grows quickly. Keep useful decisions, verified results and constraints that remain active. Summarize exchanges that no longer serve the next step while retaining a link to the original trace. Persistent memory needs a lifetime, an owner and a correction mechanism.

Anthropic suggests placing long documents early and the query later in some long-context scenarios, while structuring documents with tags. Validate that guidance on your model, language and task rather than treating it as a universal law.

Provenance, privacy and injection

Every context item needs provenance. Distinguish trusted instructions, user data, retrieved results and external content. A document may contain a malicious instruction that tries to gain priority. Nexxom’s article on prompt injection against agents describes controls when agents read untrusted sources.

The NIST AI RMF recommends identifying context, risks and responsibilities across the lifecycle. The Measure Playbook calls for histories, errors, complaints and oversight actions.

Measure the real effect

Do not measure token count alone. Track task success, correct citations, escalation rate, latency, cost and safety refusals. Compare outcome distributions when context changes. An average improvement can hide a regression in an important slice.

For RAG, separate retrieval and generation. For an agent, add tool calls and decision traces. Our article on production agent traces explains why the final answer is not enough.

Version it like software

A production context needs a version identifier. Version instructions, examples, selection rules, tool schemas, summaries and sources. Link every release to a test set and regression report. OpenAI recommends linking prompts to evals and rerunning them after publication (Prompt management).

Prepare for production

For some teams, static context and a monthly review are sufficient. For others, the agent must recalculate sources at each step and request review when critical data is missing. In both cases, set an operational confidence threshold, escalation route and owner.

Checklist

• Are the goal and next decision explicit?
• Do sources have provenance, ACL and a date?
• Are instructions separated from retrieved data?
• Is history summarized without losing critical decisions?
• Are tools limited to the current step?
• Is context versioned and linked to evals?
• Are cost, latency, errors and escalations tracked?
• Can a person remove a source or stop the agent?

Context engineering turns a token window into a governed decision system. The question is not “how do we add more context?” but “what evidence does the agent need now, with which boundaries?” For the wider method, see our work on structured outputs and hybrid RAG.

Keep evidence durable

Store prompts, model versions, selection instructions and annotation decisions with each run. Maintain a changelog for retrieval indexes, tool permissions and routing updates. When quality moves, the team can distinguish a model regression from a data or context change. This gives procurement, product and compliance teams a shared basis for expanding, limiting or pausing a deployment.

Design for handover

An agent should be able to state which source supports a decision and which required source was missing. Define a handover format that includes uncertainty, attempted tools and next human action. A concise handover is often safer than asking the model to fill a gap with plausible prose. Test handovers as part of the same release evaluation.

A release gate for context changes

Treat a context change like a production code change. Record the reason for the change, the affected source, the expected benefit and the failure mode that must remain impossible. Run the same representative task set before and after the change. Compare factual accuracy, citation coverage, tool errors, latency, token volume and human escalations. A change that improves one metric while weakening provenance is not automatically an improvement.

Use a small, explicit gate so that reviewers can make the same decision every time:

This gate also makes rollback practical. Keep the previous selector, prompt template and retrieval configuration available. If a new rule increases refusals, unsupported claims or escalations, restore the last known good configuration while the team investigates. Do not silently remove the evidence needed to explain the incident.

Make context observable without exposing secrets

Full context logging can create a privacy problem. Instead, separate operational metadata from sensitive payloads. A trace can retain source identifiers, hashes, timestamps, selection scores, tool names and output citations while storing the underlying text under an access-controlled retention policy. Redact personal data before it reaches long-lived evaluation datasets. Define who may inspect a trace, for how long and for which purpose.

The same policy should cover memory. A memory entry needs an owner, a purpose and a deletion path. Prefer explicit facts confirmed by a user or a governed system over inferred preferences. When an agent summarizes a conversation, keep the original decision reference so an operator can audit what was compressed. A summary without a reference is convenient, but it is difficult to correct when it becomes stale.

Measure the task, not the window

Token count is an input metric, not a quality outcome. Pair it with task success, groundedness, citation correctness, latency and escalation rate. For a support workflow, measure whether the answer resolves the case and cites the applicable policy. For a research workflow, measure whether the result identifies the right evidence and states uncertainty. The target context size should follow the task distribution, not a generic token budget.

When the distribution changes, revisit the selector. A context policy that worked for short requests may fail when users submit long attachments or ask the agent to call several tools. Define a capacity response in advance: summarize low-priority history, defer a non-critical tool, ask for a narrower objective or hand over to a person. This is safer than allowing the platform to truncate content without an explicit signal.