GraphRAG in Production - From Retrieval Bottlenecks to Knowledge Graphs

Introduction: The Retrieval Bottleneck

If you have heard about RAG and wondered how it works in production, this article gives you a starting point for exploring GraphRAG and knowledge graph retrieval.

Traditional RAG systems usually do not struggle most with generation. The larger bottleneck is retrieval: finding the right context before the language model writes an answer.

Similarity methods such as cosine similarity and BM25 are useful for finding relevant individual documents. They are much weaker when the answer depends on a connection spread across several documents.

There is a second problem. Even if retrieval returns one hundred relevant documents, the context window may only allow the model to see the first twenty or thirty. The remaining evidence is retrieved but never reaches the generation layer.

How GraphRAG Changes Retrieval

GraphRAG addresses these problems by turning the corpus into a queryable knowledge graph. Instead of asking the model to read a long list of independent documents, the system searches for entities and the relationships connecting them.

A knowledge graph is a database of facts represented as connections. Nodes represent entities, while edges represent relationships between those entities.

The graph gives retrieval another shape. It can start from the entities in a question, traverse the relevant neighborhood, prune unrelated branches, and pass a small connected subgraph to the language model.

Three Approaches to Building a Knowledge Graph

There is no single way to build or query a knowledge graph. Three useful approaches show different tradeoffs between global preprocessing, database queries, and infrastructure.

Clustering and Summarization

One approach groups similar nodes into clusters, summarizes each cluster, and gives those summaries to the language model. This is useful when the system needs a broad view of the entire corpus.

The cost is that clustering can become computationally expensive. Comparing many nodes with one another creates a large search space, and a separate model call may be needed to summarize every group.

It is also a global operation. The system processes information regardless of what the user asked, which means the build cost grows even when only a small part of the corpus is relevant.

Text-to-Graph Querying

A second approach translates natural language into graph database queries. The graph is explored only when a user asks a question, so there is no need to cluster the entire corpus during the build stage.

The system can use filtered database queries to find matching entities, retrieve their neighbors, and expand the graph only where relevant. This keeps the initial graph search focused on the question.

The tradeoff is operational: the application needs a separate graph structure and a database capable of storing and traversing it.

Elasticsearch as a Graph Search Engine

A third approach stores graph facts inside a text search engine. Elasticsearch can store the graph as searchable documents, which avoids introducing a separate graph database.

The graph still exists as relationships, but the application uses carefully designed filters and searches to retrieve those relationships. This makes the existing search infrastructure responsible for both text and graph retrieval.

Turning Sentences Into Triplets

The first step is converting a sentence into factual triplets. Consider this sentence:

Ben Silbermann worked at Google before founding Pinterest.

A graph representation could store it as several connected facts:

(Ben Silbermann, worked_at, Google)
(Ben Silbermann, founded, Pinterest)
(Pinterest, is_a, Startup)

There are two common ways to extract these facts. Traditional NLP uses named entity recognition, dependency parsing, and information extraction models. LLM prompting asks a language model to return facts in a strict subject, relation, object format.

Older systems relied heavily on mathematical and rule-based methods. Many modern systems now use language models for more flexible entity and relationship extraction.

Querying With Bounded Expansion

Before searching the graph, the system extracts the entities that matter in the question. For example, a question about the relationship between Nancy Pelosi and Rachida Dati produces two starting nodes.

Question: What is the link between Nancy Pelosi and Rachida Dati?
Extracted entities: Nancy Pelosi, Rachida Dati

Retrieval can then check for a direct connection, find the neighbors of each entity, and look for overlap between the two neighborhoods. If no useful overlap exists, the search expands again until a strict limit is reached.

Neighbor limits prevent the graph from exploding. A system might cap expansion at one hundred neighbors per node and three hops. Hubs connect quickly, while leaves usually add little new search space.

The worst-case search can still be large, but real graphs are usually much smaller because their topology is uneven. Most nodes have relatively few connections even when a small number of hubs are highly connected.

Pruning and Linearizing the Subgraph

Retrieving every connected node is not enough. Sending the entire subgraph to the language model can exceed the context window or add so much noise that the useful facts become hard to identify.

The first pruning pass keeps relationships that lie on the shortest paths between the query entities. If the result is still too large, a second algorithm can reduce the node and edge count further.

The result is a small focused subgraph, but a graph is not directly readable by a language model. It needs to be linearized into text.

Subject | relation | object : supporting documents
Ben Silbermann | founded | Pinterest : document_12, document_48

A practical system can describe a small number of shortest paths by chaining the relations along each path. This preserves the useful structure while presenting the evidence in a format the model can consume.

How Elasticsearch Stores and Queries the Graph

Each triplet can be stored as a document with fields such as source, destination, relation, and the sentence that supports the relationship.

{
  "source": "Ben Silbermann",
  "destination": "Pinterest",
  "relation": "founded",
  "sentence": "Ben Silbermann founded Pinterest."
}

An aggregated relation index can speed up common lookups. Filtered Boolean queries can search for triplets where the source or destination matches an entity.

Filtered KNN queries add semantic similarity while keeping the search inside a constrained subset of triplets. The result combines semantic relevance with explicit graph structure.

This design can provide fast retrieval over large document collections without requiring an LLM call during graph construction. The expensive work is moved into extraction and carefully bounded search instead of repeated global summarization.

Conclusion

Traditional RAG is strong when the answer exists inside a small number of independently relevant documents. It becomes less reliable when the answer depends on relationships spread across a corpus.

GraphRAG gives retrieval a way to follow those relationships. The important pieces are structured triplets, bounded graph expansion, focused pruning, and linearized context.

Elasticsearch shows that graph retrieval does not always require a separate graph database. With the right document shape and query constraints, an existing search engine can store, traverse, and rank the evidence needed for grounded answers.