A context window is the information a model can use during the current invocation. Memory is information preserved for later use. RAG is a retrieval process that finds external information and places selected results into the context window. They are related, but they are not interchangeable.
An effective AI system often uses all three: memory stores durable knowledge, retrieval selects what matters now, and the context window holds the active working set.
What is a context window?
The context window contains the current instructions, conversation, retrieved passages, tool results, and other inputs sent to the model. The model reasons over this active material when producing its next output.
A larger context window can hold more information, but capacity does not guarantee relevance. Loading an entire project can make the important evidence harder to distinguish from background material. Large inputs also increase latency and cost in many systems.
Use the context window for:
- The current objective and constraints.
- Recent conversational turns that still matter.
- The draft or artifact being changed.
- A focused set of retrieved sources.
- Tool output required for the next decision.
Treat it as a workbench, not permanent storage.
What is persistent memory?
Memory stores information outside the current model invocation so it can be used later. It may live in a database, file collection, vector index, knowledge graph, or specialized service.
Memory can hold different scopes. Personal memory stores stable preferences. Project memory stores sources, decisions, and examples tied to an outcome. Operational memory may store the state of a workflow or the result of a previous action.
The presence of memory does not tell the system what to retrieve. A large memory store can remain useless if it lacks good search, metadata, or provenance.
What is RAG?
Retrieval-augmented generation is a pattern in which a system retrieves relevant external information and includes it in the model's context before generation.
A simplified RAG flow is:
- Convert the question into a search query or embedding.
- Search a document collection.
- Select or rerank relevant passages.
- Add those passages to the context window.
- Generate an answer using the retrieved evidence.
RAG can retrieve from project memory, public search, a company knowledge base, or another structured source. It is a retrieval method, not a synonym for the stored knowledge itself.
| Concept | Primary job | Lifetime | Typical failure |
|---|---|---|---|
| Context window | Hold the active working set | One model invocation or conversation | Too much irrelevant material dilutes the task |
| Memory | Preserve information for later | Across sessions | Stale or untraceable information accumulates |
| RAG | Select external information for the task | Per retrieval request | The search returns incomplete or misleading passages |
How the three work together
Imagine an AI helping prepare a product launch. Project memory contains customer interviews, competitor examples, and previous positioning decisions. RAG searches that memory for evidence related to activation friction. The context window receives five passages, the current landing-page draft, and an instruction to recommend changes.
The model does not need every saved source. It needs the best evidence for the current decision.
This design also supports verification. The retrieval result can retain the source URL and passage identifier, allowing the user to inspect what informed the answer.
Where MCP fits
MCP is a protocol for connecting AI applications to external servers that expose resources and tools. A memory service can use MCP to make its search and retrieval capabilities available to compatible clients.
The MCP server overview describes resources as contextual data and tools as model-callable operations. An MCP server might expose search_project and open_source, while the client places the returned passages in its active context.
MCP does not replace memory or RAG. It provides a standard connection through which a client can use them.
Which one should you improve first?
If the answer ignores clear instructions, improve the prompt and active context. If the required information disappears between sessions, create persistent memory. If the information exists but the wrong passages are selected, improve retrieval.
Use this diagnostic order:
- Did the system store the required information?
- Did retrieval find the relevant source?
- Did the selected passage include enough context?
- Did the model receive clear instructions for using it?
- Can the user verify the result against the original source?
The distinction makes failures easier to fix. “The AI forgot” is often not one problem. It may be missing storage, weak retrieval, overloaded context, or unclear instructions.