Blog

RAG vs MCP: what each one does, and when you need both

Aperisa ยท 8 min read

Anthropic's example of a RAG chunk that lost its context is one sentence: "The company's revenue grew by 3% over the previous quarter." Which company? Which quarter? The chunk doesn't say, and a model reading it in a prompt can't ask.

That is the real difference in RAG vs MCP. RAG retrieves passages of text and puts them into the prompt before the model answers. MCP lets the model call tools, and a tool can look up a typed record and return it with a link to the document it came from. Our view: use RAG for questions about prose, and give the agent a lookup tool for any fact it will act on, such as an amount, a date or a notice period. Most agents end up with both.

Checked on 2026-10-27 against MCP spec revision 2026-07-28, the RAG paper (arXiv 2005.11401v4) and the docs under Sources. If you need the MCP basics first, start with what an MCP server is.

What does RAG actually do?

The term comes from Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (NeurIPS 2020). Their models "use the input sequence x to retrieve text documents z and use them as additional context when generating the target sequence y."

The setup was concrete. They took the December 2018 Wikipedia dump and indexed it with FAISS: "Each Wikipedia article is split into disjoint 100-word chunks, to make a total of 21M documents." A DPR retriever found the top K chunks for each input (K of 5 or 10 in training), and BART-large wrote the answer.

Two findings still matter for agents:

  • Swapping the index updates the knowledge. With a 2016 index, RAG answered 70% of questions about 2016 world leaders correctly. With a 2018 index, 68% for 2018 leaders. Mismatched, it fell to 12% and 4%. Fresh index, fresh answers.
  • Provenance was an open problem. The abstract lists "providing provenance for their decisions and updating their world knowledge" as open research problems.

Modern RAG has the same shape. OpenAI's file search splits every file into 800-token chunks with 400 tokens of overlap by default, and searches them "through semantic and keyword search." Anthropic's contextual retrieval post (September 2024) shows how much the chunk boundary costs: contextual embeddings plus contextual BM25 cut top-20 retrieval failures by 49%, and 67% with reranking.

If you run OpenClaw, you already use RAG: memory_search runs vector and keyword search over your memory files and merges the results. See where OpenClaw memory breaks for business facts.

What does MCP add that RAG doesn't?

A model that decides when to look, and a result with a shape.

The MCP spec (revision 2026-07-28) defines three server primitives and who controls each:

Primitive Control Spec description
Prompts User-controlled Interactive templates invoked by user choice
Resources Application-controlled Contextual data attached and managed by the client
Tools Model-controlled Functions exposed to the LLM to take actions

In the paper's RAG design, the retriever runs on the input and the chunks are in context before the model starts. An MCP tool is the other way around. The client calls tools/list, the model reads each tool's description and inputSchema, picks a tool and its arguments, and the client sends tools/call. Tool execution errors come back with isError: true, as "actionable feedback that language models can use to self-correct and retry with adjusted parameters."

The line is blurrier than most comparisons admit. OpenAI's file search is a hosted tool that returns chunks: RAG, invoked like a tool. The useful question is what comes back, passages to read or a record to act on.

RAG vs MCP, side by side

RAG MCP tool
What it is A retrieval technique A protocol for tools, resources and prompts
Who decides to look The pipeline, usually on every input The model, per call
What comes back Ranked text chunks Text, structured JSON, resource links, images
Sources The file the chunk came from Whatever the server attaches, such as a resource link
Freshness As fresh as the last re-index As fresh as the system the tool reads
Typical failure The right chunk ranks low, or two versions both rank The model doesn't call the tool, or calls it with bad arguments

Is MCP only for actions?

No. The spec's own definition says tools let models "interact with external systems, such as querying databases, calling APIs, or performing computations." A read-only lookup is a tool.

The 2026-07-28 spec gives that lookup a contract. A tool can declare an outputSchema, and then "Servers MUST provide structured results that conform to this schema." The result goes in structuredContent, and a tool "MAY return links to Resources," so the answer can carry the document it came from.

Here is a lookup tool for contract terms. The data is made up.

{
  "name": "get_contract_terms",
  "description": "Current terms for one client contract, with a link to the source document.",
  "inputSchema": {
    "type": "object",
    "properties": { "client": { "type": "string" } },
    "required": ["client"]
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "client": { "type": "string" },
      "notice_days": { "type": "integer" },
      "effective": { "type": "string" }
    },
    "required": ["client", "notice_days", "effective"]
  }
}

And the result of tools/call with {"client": "Harbor Lane"}:

{
  "resultType": "complete",
  "content": [
    { "type": "text", "text": "{\"client\":\"Harbor Lane\",\"notice_days\":60,\"effective\":\"2026-08-14\"}" },
    { "type": "resource_link", "uri": "https://your-server.example/docs/harbor-lane-amendment-2.pdf", "name": "Harbor Lane Amendment 2", "mimeType": "application/pdf" }
  ],
  "structuredContent": {
    "client": "Harbor Lane",
    "notice_days": 60,
    "effective": "2026-08-14"
  },
  "isError": false
}

The spec asks for the serialized JSON in a text block too, for clients that ignore structuredContent.

Now ask the same question of a RAG index over the same folder. The original contract says 30 days. The amendment says 60. Both can land in the top K, and the model has to work out which one wins. Sometimes it will. The lookup tool settled it on the server, where the effective dates are.

When do you need both RAG and MCP?

Most of the time, because they answer different questions.

RAG alone: questions about prose

"What does our refund policy say about partial months?" "Summarize the onboarding doc." A cited passage is the right answer, and nobody puts a refund policy into a schema. Below about 200,000 tokens (roughly 500 pages), Anthropic's September 2024 Contextual Retrieval post says you can skip retrieval and put the whole knowledge base in the prompt. Anthropic's current docs add that recall drops as context grows, so test it first.

A lookup tool: facts you act on

Amounts, dates, statuses, notice periods. If the agent will send a reminder or tell a client a deadline, it should get the value from a tool that returns one record and its source. Chunks are where two versions of a fact compete. On why agents guess instead, see why your bot invents client facts.

Both, behind one interface

Put your RAG search behind an MCP tool, next to the lookup tools. Every harness then gets the same retrieval instead of each one indexing its own copy. In Hermes, remote servers go under mcp_servers in ~/.hermes/config.yaml:

mcp_servers:
  records:
    url: "https://your-server.example/mcp"
    headers:
      Authorization: "Bearer ${MCP_RECORDS_API_KEY}"

Hermes resolves ${MCP_RECORDS_API_KEY} from the profile's secrets or the process environment, so the token stays out of the file. Run /reload-mcp in a session to pick it up. In OpenClaw, under mcp.servers in ~/.openclaw/openclaw.json:

{
  mcp: {
    servers: {
      records: {
        url: "https://your-server.example/mcp",
        transport: "streamable-http",
        headers: { Authorization: "Bearer ${MCP_TOKEN}" },
      },
    },
  },
}

Set transport explicitly, since OpenClaw uses SSE when it's omitted. Then check the connection with openclaw mcp doctor records --probe. OpenClaw's docs say to keep credentials out of config literals, and the ${VAR} reference does that in both harnesses.

Use OpenClaw 2026.5.12 or later and Hermes Agent 0.19.0 or later. Older OpenClaw builds could send your Authorization header to another site on a redirect (advisory), and older Hermes builds have a supply chain issue in the bundled MCP catalog (advisory).

Then tell the agent which to use:

Amounts, dates, notice periods, statuses: call get_contract_terms (or the
matching get_ tool) and quote the source link. Use search for questions
about what a document says. If neither returns anything, say so.

Where does Aperisa fit?

Aperisa is a remote MCP server. Your data is available to any harness that connects over Streamable HTTP with a Bearer key: OpenClaw, Hermes, Cursor, Codex, Antigravity, n8n or your own code.

Give each agent its own key and set what it can reach: which data areas, work or personal, and how much detail, from full records down to totals only. Aperisa enforces those limits on the server.

Chunks for reading, records for acting

RAG finds the right passage. An MCP tool lets the model ask for something, and that something can be a typed record with a link to its source. Use chunks when the answer is a paragraph, and a lookup when it's a number someone will act on.

Aperisa connects as a remote MCP server over Streamable HTTP with a Bearer key, and each key can be limited by data area and level of detail. Try it out.

FAQ

Is MCP a replacement for RAG? No. RAG is a retrieval technique: it finds text chunks and puts them into the prompt. MCP is a protocol for exposing tools, resources and prompts to an AI application. A RAG search can sit behind an MCP tool, and many setups use both.

Can an MCP tool return structured data? Yes. Since the 2025-06-18 revision, and still in the 2026-07-28 spec, a tool can declare an outputSchema and return a JSON value in structuredContent. It can also return resource links that point to the source document.

When is RAG enough on its own? When the questions are about prose, such as policies, docs or meeting notes, and a paraphrase with a cited passage is a good answer. Anthropic's 2024 guidance is that a knowledge base under about 200,000 tokens can go straight into the prompt with no retrieval at all.

Does MCP stop an agent from using stale facts? Not by itself. MCP only defines how a client calls the tool. Freshness depends on what the server behind the tool reads, and whether the agent is told to call it instead of answering from memory or old chunks.

Sources

Questions

Is MCP a replacement for RAG?

No. RAG is a retrieval technique: it finds text chunks and puts them into the prompt. MCP is a protocol for exposing tools, resources and prompts to an AI application. A RAG search can sit behind an MCP tool, and many setups use both.

Can an MCP tool return structured data?

Yes. Since the 2025-06-18 revision, and still in the 2026-07-28 spec, a tool can declare an outputSchema and return a JSON value in structuredContent. It can also return resource links that point to the source document.

When is RAG enough on its own?

When the questions are about prose, such as policies, docs or meeting notes, and a paraphrase with a cited passage is a good answer. Anthropic's 2024 guidance is that a knowledge base under about 200,000 tokens can go straight into the prompt with no retrieval at all.

Does MCP stop an agent from using stale facts?

Not by itself. MCP only defines how a client calls the tool. Freshness depends on what the server behind the tool reads, and whether the agent is told to call it instead of answering from memory or old chunks.

Published

Keep reading