Is RAG Dead?

RAG is dead? Let's think about this.

Google released Gemini with 1M tokens, then 2M. Meta dropped Llama 4 with a 10M context window. They achieved this with the iRoPE architecture — two types of attention in the LLM:

1. Local Attention: For short contexts — the classic scenario where LLMs naturally look at their allowed context.
2. Global Attention: For long contexts that go beyond the allowed limit, but we still want the LLM to pay attention to.

Imagine a word processor that can only see 100 lines at once. Most LLMs work similarly — a fixed buffer (e.g., 8K tokens). Now imagine making it work with a 10 million line document without losing context of the first pages. That's the "infinity context window."

RoPE (Rotary Position Embedding) uses rotation to give order to vectors, making understanding data at scale faster and more efficient for transformer architectures. The "i" stands for "interleaved layers, infinite."

But does this make RAG obsolete?

Although having a model capable of handling large amounts of context is a giant leap forward, RAG remains useful for external enrichment in generation — especially when up-to-date or highly specific data is required that can't be stored within a single context due to its diversity. It's also cost-efficient and more optimal.

I believe we'll see a hybrid model between Long Context and RAG — something we haven't seen before. Almost no limits for the model to access very extensive data, in an efficient, fast, and cost-optimal manner.

RAG won't die. It will evolve.