GraphRAG vs Vector RAG: Choose by Question Shape
Start with vector RAG for passage-level lookup and local semantic matching. Consider GraphRAG when evaluated questions depend on entities, relationships, communities, or themes spanning many documents—and only when the extra extraction, indexing, query, and refresh work earns its place.
Match architecture to the information need
| Question shape | Start with | Why | Evidence to demand |
|---|---|---|---|
| Find the clause describing one policy | Vector RAG | The answer is passage-local. | Required passage recall and rank. |
| Explain how two named entities are connected | GraphRAG candidate | Explicit relationships may organize retrieval. | Correct entities, edges, and source passages. |
| Summarize themes across the whole corpus | GraphRAG global query candidate | Corpus-level community summaries target broad questions. | Theme coverage, source traceability, and cost. |
| Answer many narrow support questions | Vector RAG | Simple passage retrieval may meet the task with less indexing work. | Latency, freshness, and answer grounding. |
| Investigate a specific entity with local context | GraphRAG local query candidate | Entity neighborhood plus text can expand evidence. | Neighborhood precision and missing relationships. |
| Unproven relationship requirement | Vector baseline | A graph should answer a measured failure, not an imagined future need. | Baseline misses classified by question type. |
GraphRAG changes the index, not only the query
Vector RAG typically parses documents into chunks, embeds them, and retrieves nearby passages for a question. The original RAG paper describes a dense vector index as non-parametric memory for the generator. Production implementations add filters, lexical search, reranking, and citation handling, but the primary retrieval unit remains a document passage.
GraphRAG adds an extraction and summarization pipeline. Microsoft's official indexing overview documents outputs that support entity, relationship, community, and text-unit query paths. That creates new versioned artifacts and failure boundaries: extraction prompts, entity resolution, relationship claims, communities, summaries, and links back to source text.
Freshness is now a graph-maintenance question
When a document changes, a passage index can re-embed affected chunks. A graph pipeline may also need to revise entities, relationships, communities, and derived reports. Define incremental behavior, deletion propagation, stale derived artifacts, and a rebuild threshold before choosing the architecture.
Separate retrieval mode from answer generation
Microsoft's GraphRAG query overview describes local search as combining graph data with raw text chunks for entity-focused questions. It describes global search as operating over community reports in a map-reduce style and notes that it is resource intensive. It also includes a basic vector RAG path for comparison.
Those query methods retrieve or synthesize context; they do not make the final answer automatically correct. Preserve the source text behind entities, relationships, and reports. The generator must distinguish extracted claims from primary passages, surface conflicts, cite eligible documents, and abstain when a path in the graph lacks adequate source support.
Evaluate the question slices that justify a graph
Build distinct slices for passage lookup, entity facts, direct relationships, multi-hop relationships, neighborhood exploration, and corpus-wide themes. Compare vector RAG and the relevant GraphRAG query mode on the same source snapshot. Measure evidence coverage, relationship correctness, citation support, task usefulness, latency, index work, query work, and refresh behavior.
Use ablations. Compare GraphRAG context with and without graph-derived material, and compare against vector passages alone. If the graph adds fluent summaries but no better supported answer, it has not earned the operating complexity. The RAG evaluation metrics guide provides the retrieval-versus-generation boundary needed for this comparison.
Failure cases that change the decision
- Extraction fiction: an incorrect entity or relationship becomes a durable graph fact. Retain source spans and confidence or review status.
- Entity collision: two people or products with the same name merge. Test aliases, abbreviations, and ambiguous names.
- Orphaned relationship: an edge exists without adequate passage support. Require traceable evidence before generation cites it.
- Global summary drift: updated documents do not refresh community reports. Version derived artifacts and test deletion propagation.
- Graph for local lookup: a complex path adds cost but returns the same passage as vector search. Route narrow questions to the simpler method.
- Authorization through edges: a relationship reveals restricted entity information. Apply policy to graph nodes, edges, reports, and source text.
A decision-first implementation sequence
- Classify question shapes. Separate local passage, entity, relationship, multi-hop, neighborhood, and corpus-wide needs.
- Build the vector baseline. Establish chunking, filters, retrieval, grounding, citations, latency, and refresh behavior.
- Identify unrepaired failures. Confirm that graph-worthy misses remain after reasonable query, hybrid search, and reranking work.
- Create a bounded graph pilot. Use one representative corpus, version every extraction artifact, and preserve links to source spans.
- Test the matching query modes. Compare local or global GraphRAG only on the questions each mode is intended to serve.
- Measure full lifecycle cost. Include extraction, model calls, storage, refresh, deletion, query latency, observability, and incident repair.
- Route selectively or stop. Keep GraphRAG only for slices with a supported gain and retain vector RAG as the simpler path where it passes.
Primary and official sources
- Microsoft GraphRAG indexing overview — official documentation for graph-oriented indexing artifacts.
- Microsoft GraphRAG query overview — official local, global, DRIFT, and basic search descriptions.
- Lewis et al., original RAG paper — the dense retrieval and generation baseline.