GraphRAG is retrieval-augmented generation that uses a knowledge graph — data represented as entities connected by relationships — often alongside vector search. It lets a model follow relationships and answer multi-hop and corpus-wide questions that plain vector RAG handles poorly.

When should you use a knowledge graph instead of vector RAG?

Use a knowledge graph for multi-hop reasoning, connecting information across many documents, and global-summarization questions. Use vector RAG for simple factual lookups. Many production systems combine both. Graphs add significant indexing cost, so use them when questions are genuinely relational.

What is a context graph?

A context graph applies the knowledge-graph idea to an agent’s working knowledge — representing what the agent knows as connected entities and relationships rather than flat text. It helps an agent reason over relationships in its context rather than just similarity.

Knowledge Graphs for RAG (GraphRAG): When Graphs Beat Vectors

Vector RAG is great at finding the most similar chunk. It is poor at questions that span many documents or follow relationships across them. That is where knowledge graphs — and GraphRAG — earn their (considerable) cost. Here is when graphs beat vectors, and when they are overkill.

8 min read·By Bryan Rodrigues

Knowledge graphs, GraphRAG, and context graphs

A knowledge graph represents your data as entities (nodes) connected by labelled relationships (edges) — Customer places Order contains Product supplied by Supplier. GraphRAG is retrieval-augmented generation that uses such a graph, often alongside vector search, so the model can follow relationships rather than just match similar text. A "context graph" is the same idea applied to an agent’s working knowledge: representing what it knows as connected entities rather than a flat blob of text.

A knowledge graph makes relationships explicit, so a query can traverse from one entity to another across multiple hops.

When graphs beat vectors

The advantage shows up on questions plain vector RAG handles badly. Multi-hop queries — "how is Person A connected to Event B through Organisation C?" — are native to a graph but become three disconnected lookups for vector search. Global-summarization questions — "what are the main themes across all these documents?" — defeat vector RAG, which can only fetch a handful of chunks; graph community summaries capture dataset-wide patterns. Published benchmarks have shown vector-only RAG failing badly on schema-bound queries (KPIs, forecasts) where graph approaches reach high accuracy.

●Use vector RAG for simple factual lookups — "what is the refund policy?" A graph is unnecessary overhead here.
●Use GraphRAG for multi-hop reasoning, connecting disparate sources, and global summarization across a corpus.
●Most mature enterprise systems combine both — graph traversal plus semantic search.

The cost trade-off

Graphs are not free. Building the graph (indexing) can cost orders of magnitude more than vector indexing — reports cite 100–1000x for full GraphRAG — though smarter variants like LazyGraphRAG cut that dramatically. The honest rule: do not reach for a knowledge graph because it sounds sophisticated. Reach for it when your questions are genuinely relational or corpus-wide, and the value justifies the indexing cost.

Sources