A manager asks an AI agent why a project is running late. The answer arrives quickly: the supplier has missed deadlines, several tasks remain open, and the delivery plan needs updating. There are even links to sources. But opening them reveals a problem. The delivery date comes from a superseded contract, some unfinished tasks belong to the next phase, and the account blaming the supplier predates an approved scope change.
This is a fictional example, but it illustrates a real engineering question. The model may have read every paragraph correctly. The search may have found documents about exactly the right project. Yet the answer still misrepresents its status. To understand why, we need to examine how the picture presented to the model was assembled.
I propose treating context as a component of the architecture: something with an explicit construction process, rules it must satisfy, and tests. Those decisions shape the answer before the model starts writing it.
What the model actually receives
Connecting a model to company systems does not automatically turn their contents into its internal knowledge. Each call receives particular inputs: instructions, the question, conversation history, retrieved documents and tool results. In an agent with several steps, those inputs change between calls. This distinction also underlies Anthropic’s account of context engineering.
Someone must decide which project the user means, the date of the assessment, which systems hold the necessary information, and what happens when one is unavailable. Supplying the first ten search results is itself a policy. It delegates much of the construction of the model’s picture of reality to the ranking mechanism.
Our question contains two tasks: establish whether the project is expected to miss a valid commitment, and explain the reasons. The first requires precise dates and definitions. The second requires evidence linking events. Combining them too early lets a system produce a persuasive explanation before establishing that a delay exists.
First establish the baseline
Suppose the original contract specified delivery on October 15. An approved amendment, effective October 12, moved delivery to November 5. The task system still shows the original date. A supplier message dated October 20 forecasts delivery on November 12. We are assessing the situation on November 2.
The accurate answer is a forecast seven-day variance against the approved date, assuming this is the latest forecast and no further change has been approved. Delivery is not yet overdue against the revised commitment. The 28-day difference from the original date is another measure. It may matter when reviewing project performance, but it cannot substitute for comparison with the current commitment.
Sorting files from newest to oldest will not resolve this. Upload date, creation date and effective date can differ. Yesterday’s draft does not necessarily supersede an amendment approved three weeks ago. Nor does “approved” in a filename establish its status. The system needs an authoritative approval record, or it must leave that question unresolved.
I would define an explicit business rule: take the commitment date from approved change records, the delivery forecast from the designated reporting source, and calculate the variance in code after validating the date type and milestone. Where sources conflict and no rule settles the issue, return the conflict. The model can explain it; it should not invent the organization’s hierarchy of authority.
Different information needs different retrieval
Semantic search is useful when looking for a discussion of changing requirements or a narrative explanation of a defect. For a list of a project’s open tasks, I would start with a query to the task system, constrained by project identifier, milestone and update time. Turning a table into documents and asking search to reconstruct the relevant rows offers no obvious advantage.
Structured data can be filtered and calculated. Documents need to be located and read. Binding decisions require verification against their approval process. A model can help formulate a query and interpret its results, but the tool interface should constrain and validate its parameters. Joining on a stable project identifier also helps avoid combining two projects with similar names.
Within document search, semantic matching can find differently worded expressions of an idea, while lexical search helps with contract numbers or error codes. Anthropic illustrates their complementary roles and the problem of fragments losing their surroundings in Contextual Retrieval. In our example, “the date was moved by two weeks” is insufficient without the milestone and whether this was a proposal or a decision.
I would therefore return a source identifier, version, project and section alongside each excerpt, with a way to open the full passage. An automatically generated summary can help locate material. When the conclusion depends on “subject to approval,” the original wording matters. A high relevance score does not establish authority to set a deadline.
From dates to causes
Calculating the forecast variance still does not explain it. Suppose three supplier tasks are marked late. Do they block delivery? Two might belong to the next phase; the third might have schedule float. Connecting them to the delay requires examining dependencies and their effect on the milestone, rather than counting red tasks.
I would ask the tool for the relevant dependency chain, schedule changes and reports explaining those changes. If a reliable schedule exists, dependency effects should be calculated in an appropriate scheduling tool. Without one, a language-based estimate should not be presented as a critical-path calculation.
We must also distinguish a fact, a stakeholder’s explanation and our own conclusion. “The supplier reported that testing was postponed because of an interface change” is attributable to a particular message. “The interface change caused the entire delay” is a broader conclusion. It requires checking when the change was approved, whether testing actually depended on it, and what happened concurrently.
A careful answer can establish a forecast overrun, present the supplier’s explanation, and state that the available evidence does not attribute the entire variance to that explanation. The output should make these distinctions visible: what is known, which source supports each claim, and what remains unresolved. A link at the end of a paragraph is insufficient if it supports only half of it.
Permissions also constrain conclusions
A user may have access to the task system but not the contract amendment. Sending the amendment to the model and instructing it not to reveal it is an inadequate design. In the architecture proposed here, access checks happen before content reaches the model or an external processing service. Identity comes from the authenticated application, not a parameter the model can choose.
Microsoft’s documentation on security filtering illustrates a useful distinction: filtering by a user or group identifier does not itself authenticate the requester. The application must correctly connect authenticated identity to the filter.
This also changes what can be concluded. A missing document in the results does not establish that no decision was made. Without an accessible basis for identifying the binding date, the system must limit its answer to what it can verify, without disclosing the existence of confidential material. Cached answers must respect the same boundaries: an answer assembled for one manager may not be suitable for another.
Content and instructions require a similar separation. A supplier’s message is evidence to examine, even if it tells the agent to ignore another document or send a report to an address. Reading it grants no authority to act. Permission to update a system or transmit information must be checked independently.
What to retain, and what not to compress away
We could begin with every project document, but that is not always a good choice for latency, cost or understanding what influenced an answer. Here, I would start with milestone status, the valid commitment and the latest forecast, expanding the investigation when a discrepancy appears. The design also needs stopping conditions: after how much time or how many attempts should it return a partial answer with a specific unresolved gap?
This does not mean longer context is always harmful. For a small project, reading a few complete documents may be simpler and more reliable than splitting, indexing and retrieving them through several stages. The choice needs testing on the actual task. Another tool call may improve the evidence, while adding latency and another service dependency.
Memory between assessments introduces a different risk. A summary saying “the project is delayed because of the supplier” can persist weeks after circumstances change. I would preserve the assessment date, claims, sources and qualifications. A forecast must remain a forecast; a conditional decision must remain conditional. The next assessment should refresh changeable data rather than treat yesterday’s generated answer as an independent source.
Testing the architecture
Judging the prose alone makes failures hard to locate. An approved amendment that was never retrieved presents a different problem from one the model received and ignored. If the task query mixed two milestones, replacing the model may leave the underlying input error untouched.
I would build test cases with expected outcomes and annotated evidence. One contains an old contract and a valid amendment; another contains a newer, unapproved draft. A third lacks information needed for a conclusion, and a fourth restricts the user’s access. Include stale sources, unavailable tools and instructions embedded in retrieved documents.
For each case, run two tests with the same model and instructions. In the first, let the system gather the evidence. In the second, provide the correct, authorized evidence directly. If the second succeeds and the first fails, investigate retrieval and context assembly. If both fail, examine instructions, evidence presentation and the model’s ability to reason from it. This is not perfect isolation, but it reduces guesswork.
Measure what the task requires: selection of the valid date, correct variance, support for causal claims and restraint when evidence is insufficient. Measure latency and cost alongside these. Diagnosis needs traces of source versions, tool calls and failures, with restricted log access and minimal unnecessary copying of sensitive content.
This is where I would start implementation: one recurring question, known sources, explicit decision rules and failure cases. Once we can explain why the system succeeded and why it failed, there is a basis for expanding it. When someone asks why a project is delayed, we should be able to inspect both the answer and the construction of the picture behind it.
The project, dates and scenario are illustrative. The design choices are proposals, not claims about a tested deployment. Primary technical sources are linked in the article.