GraphRAG: extraction prompts, communities, local vs global search

What it is

GraphRAG builds a knowledge graph from a document corpus at index time, then answers questions by traversing that graph rather than (or as well as) retrieving passages by similarity.

The indexing pipeline:

documents
   │
   ▼  1. chunk
   │
   ▼  2. EXTRACT entities and relationships with an LLM, per chunk
   │        ("Acme Corp" --acquired--> "Beta Ltd", 2024)
   │
   ▼  3. RESOLVE duplicates across chunks
   │        ("Acme Corp" == "Acme" == "ACME Corporation")
   │
   ▼  4. DETECT communities (clusters of densely connected entities)
   │
   ▼  5. SUMMARISE each community with an LLM
   │
   ▼  graph + community summaries, ready to query

And two query modes that are genuinely different operations:

Mode	Question shape	Mechanism
Local search	"What did Acme acquire in 2024?"	Find the entity, traverse its neighbourhood, gather the connected chunks
Global search	"What are the main themes across these documents?"	Map over community summaries, reduce into an answer

What it is confused with: vector RAG with a graph database attached. Storing embeddings in Neo4j is not GraphRAG. The defining property is that retrieval follows relationships extracted from the text, so a question whose answer requires connecting facts stated in different documents can be answered, which similarity search structurally cannot do.

The other confusion: GraphRAG is expensive at index time in a way vector RAG is not. Every chunk gets at least one LLM call for extraction, plus community summarisation. That cost is the central fact of the technique and it decides most of the adoption question.

The problem it solves

Vector retrieval finds passages similar to the query. Two question types defeat it structurally, and knowing which is which is the whole decision.

Multi-hop questions. "Which of our suppliers are also customers of our main competitor?" No single passage says that. The answer requires joining supplier records to competitor customer records through company identity, and a passage about a supplier is not similar to a passage about a competitor's customers.

Vector RAG:  retrieves passages about suppliers, and passages about the
             competitor, and the model must join them if both happen to be
             in the top-k. Usually one or the other dominates.
GraphRAG:    traverse Supplier --supplies--> Us, and Company --customer_of-->
             Competitor, intersect on Company. The join is in the index.

Global questions. "What are the recurring themes in these 4,000 support tickets?" There is no passage that answers it, because the answer is a property of the corpus rather than of any document. Top-k retrieval returns k tickets, which is a sample, not a summary.

Vector RAG:  returns 10 tickets. The model summarises 10 tickets and calls
             them the themes. This is wrong in a way that looks right.
GraphRAG:    community summaries already describe clusters across the whole
             corpus; the query maps over them and reduces.

The measured difference on global questions is large, and it is the case GraphRAG was designed for. Microsoft's evaluation reported substantial win rates for GraphRAG over vector RAG on comprehensiveness and diversity for corpus-level questions, and roughly parity on simple fact lookup.

Mechanics

Extraction: the prompt is the schema

Given a text document, identify all entities and the relationships between them.

For each entity, extract:
  - entity_name: capitalised
  - entity_type: one of [PERSON, ORGANISATION, PRODUCT, LOCATION, EVENT]
  - entity_description: a comprehensive description of its attributes and
    activities AS STATED IN THIS TEXT

For each pair of clearly related entities, extract:
  - source_entity, target_entity
  - relationship_description: why you believe they are related
  - relationship_strength: 1-10

Return as a delimited list. Text: {chunk}

Three design decisions in that prompt matter more than they look:

A closed entity-type list. Open-ended typing produces COMPANY, ORGANIZATION, ORG, BUSINESS for the same concept and the graph fragments. Enumerating types is the cheapest quality lever available.

Descriptions rather than bare names. The description is what makes resolution and summarisation possible later. Two "Acme" entities with descriptions can be judged the same or different; two bare names cannot.

"As stated in this text." Without it the model supplies world knowledge, and the graph contains relationships that are not in your corpus. That is a correctness problem that looks like recall.

Gleaning is the standard refinement: after the first extraction pass, ask "did you miss any entities?" and merge. Microsoft's implementation defaults to one gleaning round and reports meaningful recall improvement; each round is another LLM call per chunk, so it is a direct cost multiplier.

Resolution: the step that determines whether the graph works

Extraction produces Acme Corp, Acme, ACME Corporation, Acme Corp. as four entities from four chunks. Unresolved, the graph has four nodes where it should have one, and every traversal through Acme finds a quarter of its edges.

This is entity resolution as a full topic; the GraphRAG-specific shape is:

def resolve(entities: list[Entity]) -> list[Entity]:
    # 1. Blocking: only compare plausible candidates.
    blocks = group_by(entities, key=lambda e: (e.type, normalise(e.name)[:4]))

    merged = []
    for block in blocks.values():
        # 2. Cheap similarity first.
        clusters = cluster(block, sim=lambda a, b:
            0.5 * name_similarity(a.name, b.name) +
            0.5 * cosine(a.description_emb, b.description_emb))
        # 3. LLM adjudication only for the ambiguous middle.
        for c in clusters:
            if c.confidence < 0.85:
                c = llm_adjudicate(c)
            merged.append(merge_entity(c))     # descriptions are CONCATENATED
    return merged

Merging descriptions rather than picking one is important: the merged entity should carry every chunk's account of it, because that combined description is what the community summariser reads.

Communities and summarisation

Run a community detection algorithm (Leiden, or Louvain) over the resolved graph. Leiden produces a hierarchy: fine communities at level 0, coarser at each level up.

Level 0:  47 communities, 8-30 entities each   (specific: one product line)
Level 1:  12 communities, 40-120 entities      (a business unit)
Level 2:   4 communities, 200-600 entities     (a market segment)

Each community is summarised by an LLM, bottom-up: level-0 summaries feed level-1 summarisation, and so on. The hierarchy is what makes global search tractable: a global question maps over the level-1 or level-2 summaries (a few dozen) rather than over every entity.

def summarise_community(community, level, child_summaries=None):
    if level == 0:
        content = format(community.entities, community.relationships)
    else:
        content = format(child_summaries)     # summarise the summaries
    return model(COMMUNITY_SUMMARY_PROMPT + content)

Local search

def local_search(query: str, k_entities: int = 10) -> str:
    seeds = entity_index.search(query, k=k_entities)     # vector search over entities
    subgraph = graph.neighbourhood(seeds, hops=2)

    context = assemble(
        entities=subgraph.entities,                       # with descriptions
        relationships=subgraph.relationships,
        source_chunks=chunks_for(subgraph),               # the original text
        community_summaries=[c.summary for c in subgraph.communities])
    return model(LOCAL_SEARCH_PROMPT + context + query)

Note that the source chunks are included. Local search is vector RAG plus the neighbourhood: you still give the model the original text, and the graph decides which text by relationship rather than by similarity alone. That is why local search rarely does worse than vector RAG and often does better on multi-hop.

Global search: map-reduce over communities

def global_search(query: str, level: int = 1) -> str:
    communities = graph.communities_at_level(level)

    # MAP: each community summary answers the question independently,
    # returning points with a self-assessed score.
    partials = parallel_map(communities, lambda c:
        model(MAP_PROMPT + c.summary + query, schema=RatedPoints))

    # Keep only what the map step rated as relevant, ordered.
    points = sorted([p for r in partials for p in r.points
                     if p.score > 0], key=lambda p: -p.score)[:TOP_N]

    # REDUCE: synthesise the surviving points.
    return model(REDUCE_PROMPT + format(points) + query)

Cost is proportional to community count, so a global query at level 1 with 12 communities is 13 model calls. At level 0 with 47 it is 48. Level selection is the cost/detail dial and it should be exposed rather than fixed.

The index-time cost, which is the deciding number

Corpus: 10,000 documents, ~600 chunks of 1,200 tokens each per 1,000 docs
        = 6,000 chunks

Extraction:      6,000 chunks x ~1,800 tokens in / 900 out
                 + 1 gleaning round (another 6,000 calls)
                 = 12,000 LLM calls

Resolution:      ~2,000 LLM adjudications for ambiguous pairs

Community
  summarisation: 47 + 12 + 4 = 63 summaries, some over large inputs

Total:           ~14,000 LLM calls, roughly 32M input / 14M output tokens

At mid-tier model prices that is a few hundred dollars and several hours for 10,000 documents. Vector RAG over the same corpus is 6,000 embedding calls, a few dollars and a few minutes.

Roughly two orders of magnitude more expensive to index. And it must be redone, at least partially, when documents change.

A worked example: 4,000 support tickets

A B2B SaaS company wanted two things from its ticket archive: answer specific historical questions, and understand recurring problems.

Baseline: vector RAG over 4,200 tickets.

index cost:            ~$4, 6 minutes
query cost:            $0.011
p50 latency:           1.2s

Specific questions ("what was the resolution for the Contoso SSO issue?")
  accuracy:            84%

Multi-hop ("which customers reported both SSO and SCIM problems?")
  accuracy:            31%

Global ("what are the top recurring integration problems?")
  usefulness (human):  2.1/5

The global answer was the visible failure. Asked for recurring themes, it retrieved 10 tickets and summarised those 10, producing a confident answer that described whichever 10 tickets happened to be most similar to the word "recurring." Wrong in a way that looks right, which is the worst failure profile.

GraphRAG index build:

entity types:          [CUSTOMER, PRODUCT_FEATURE, INTEGRATION, ERROR_TYPE, PERSON]
chunks:                4,200 (one ticket per chunk, mostly)
extraction:            4,200 calls + 4,200 gleaning
entities extracted:    18,400 raw
after resolution:      3,100
relationships:         11,200
communities (Leiden):  L0=38, L1=9, L2=3

index cost:            $310
index time:            4h 10m

Results:

                          vector RAG    GraphRAG local    GraphRAG global
specific questions          84%            86%                n/a
multi-hop                   31%            79%                n/a
global usefulness           2.1/5          n/a                4.3/5
query cost                  $0.011         $0.019             $0.31
p50 latency                 1.2s           2.1s               14s

Multi-hop went from 31 to 79 percent and global from 2.1 to 4.3 out of 5. Specific lookup was essentially unchanged, which is the expected shape: GraphRAG does not help where similarity already works.

Global queries cost $0.31 and 14 seconds, because a level-1 global query is 10 model calls fanned out and one reduce. That is fine for an analyst asking a few questions a day and unusable in an interactive product.

Three problems in the first month.

Problem 1: entity type drift. The initial extraction prompt did not enumerate types.

entity types produced:   340 distinct
examples:                CUSTOMER, CLIENT, ACCOUNT, ORGANISATION, COMPANY,
                         CUSTOMER_ORG, END_USER  (all the same thing)
graph connectivity:      poor; traversals died at type boundaries
multi-hop accuracy:      52%

Closing the type list to five took multi-hop from 52 to 79 percent. Enumerating entity types was the single largest quality lever in the whole build, and it is one line of prompt.

Problem 2: resolution recall. The first resolution pass used exact-normalised-name matching only.

entities after resolution:   9,800 (should have been ~3,000)
"Contoso" variants:          Contoso, Contoso Ltd, Contoso Limited,
                             contoso, Contoso EMEA  -> 5 separate nodes

Each variant carried a fraction of the edges, so a question about Contoso found a fifth of what was known. Adding description-embedding similarity plus LLM adjudication for the 0.6 to 0.85 confidence band took it to 3,100 entities and was worth 14 points of multi-hop accuracy.

Problem 3: incremental updates. New tickets arrived at about 60 a day, and the initial design rebuilt the whole index nightly.

nightly rebuild cost:    $310
monthly:                 $9,300     <- unacceptable

def incremental_update(new_chunks):
    new_entities = extract(new_chunks)                        # only new chunks
    resolved = resolve_against_existing(new_entities, graph)  # not a full re-resolve
    graph.merge(resolved)

    # Only re-detect communities and re-summarise the AFFECTED ones.
    touched = graph.communities_containing(resolved.entity_ids)
    if graph.modularity_drift() > REDETECT_THRESHOLD:
        recompute_communities()                                # rare, full
    else:
        for c in touched:
            c.summary = summarise_community(c)                 # partial

daily incremental cost:  $310 -> $6.20
full rebuild:            monthly, or on modularity drift

Incremental update is the difference between GraphRAG being affordable and not, and it is the part most implementations lack. Community structure is the awkward piece: adding entities can change cluster membership, so you need a drift measure to decide when a full re-detection is due.

Final architecture, and it uses both systems:

def answer(query: str) -> str:
    kind = classify(query)                    # a small, cheap classifier
    if kind == "global":
        return graphrag_global(query, level=1)      # $0.31, 14s
    if kind == "multi_hop":
        return graphrag_local(query)                # $0.019, 2.1s
    return vector_rag(query)                        # $0.011, 1.2s

                        before      after
specific accuracy         84%        84%
multi-hop accuracy        31%        79%
global usefulness         2.1/5      4.3/5
mean query cost           $0.011     $0.024   (weighted by actual mix)
p50 latency               1.2s       1.4s
index cost                $4/mo      $190/mo  (incremental + monthly rebuild)

Routing by question type is what made it economic. Eighty percent of queries were specific lookups that vector RAG answers well at a twentieth of the cost, so paying GraphRAG prices for all of them would have been indefensible. GraphRAG is a capability to route to, not a replacement.

Production evidence

Microsoft Research published GraphRAG in 2024 with an open-source implementation, and their evaluation on podcast transcripts and news articles reported GraphRAG winning substantially on comprehensiveness and diversity for corpus-level "sensemaking" questions, while being comparable on simple retrieval. Their paper is explicit that the target is global questions that vector RAG cannot address in principle.

The Microsoft implementation defaults to Leiden for community detection and produces a hierarchy, which is what makes global search cost-tunable by level.

LlamaIndex, Neo4j and LangChain all ship GraphRAG-style implementations, and Neo4j's in particular is worth knowing because it treats the graph as a first-class queryable store so you can write Cypher against the extracted graph, which is a capability the summarise-and-retrieve implementations lack.

"LazyGraphRAG" (Microsoft, 2025) defers the expensive summarisation until query time, reporting substantially lower index cost with comparable quality on many questions. It is the direct response to the index-cost objection and worth naming, because "GraphRAG is too expensive to index" is now a dated criticism.

Writer's and several vendors' knowledge-graph RAG offerings target the same multi-hop gap, and the consistent pattern across published results is large gains on multi-hop and global questions with little or no gain on single-fact lookup.

The debate

Is GraphRAG worth it? Only for the question types it addresses, and the honest framing is that it is roughly two orders of magnitude more expensive to index for a capability most queries do not need. My position: build vector RAG first, measure your question mix, and add GraphRAG only if multi-hop and global questions are a meaningful fraction. In the worked example they were about 20 percent, and routing meant the other 80 percent stayed cheap.

What is the actual failure of vector RAG on global questions? It answers them confidently and wrongly. Retrieving ten tickets and summarising them produces a fluent answer describing ten tickets, presented as the themes of four thousand. That is worse than a refusal, because there is no signal it is a sample. If you take one thing from this page, it is that "what are the main themes" is not a retrieval question and vector RAG will not tell you it cannot answer it.

Extraction quality: what dominates? The entity type list, by a wide margin. Open-ended typing produced 340 types where 5 were meant, traversals died at type boundaries, and multi-hop accuracy was 52 percent instead of 79. Enumerate the types, and choose them from your domain rather than using generic ones, because INTEGRATION and ERROR_TYPE carry more structure for a support corpus than THING.

Is resolution the hard part? Yes, and it is under-invested. Five unresolved variants of one company means five nodes each holding a fifth of the edges, so the graph is present and the traversals fail. It is worth more engineering than extraction, and cheap similarity before expensive LLM adjudication is the shape that makes it affordable. See entity resolution.

Can you afford to keep it fresh? Only with incremental update, and community structure is what makes that awkward: new entities can change cluster membership, so you need a drift measure and a periodic full re-detection. A GraphRAG system without incremental update is a snapshot, and for a corpus that changes daily that is usually not what was wanted.

Graph database or not? For traversal-only use, a graph in Parquet or a document store is sufficient and simpler. A real graph database (Neo4j, Kuzu) earns its place when you want to write queries against the extracted graph directly, which is a genuine capability: "list every customer connected to more than three error types" is a Cypher query and not an LLM question at all. That analytical use is under-exploited and is sometimes worth more than the RAG improvement.

Follow-up Q&A

"When does GraphRAG beat vector RAG?"

Two question shapes. Multi-hop, where the answer requires joining facts stated in different documents, because no single passage is similar to the query and similarity search cannot perform the join. And global, where the answer is a property of the corpus rather than of any document, because top-k returns a sample and the model summarises the sample as though it were the whole. On simple fact lookup they are comparable, which is most queries, so the right architecture routes by question type.

"Why does vector RAG fail on 'what are the main themes'?"

It retrieves ten passages and the model summarises those ten, producing a fluent confident answer that describes ten documents out of four thousand. Nothing in the output signals that it is a sample. That is a worse failure than a refusal, and it is structural rather than a tuning problem: there is no passage in the corpus that answers a corpus-level question, so retrieval by similarity cannot find one.

"How does global search work?"

Map-reduce over community summaries. Community detection (Leiden) produces a hierarchy of entity clusters; each community is summarised bottom-up. A global query sends the question to every community summary at a chosen level, each returns rated points, the points are ranked and the top ones are synthesised. Cost is proportional to community count, so the hierarchy level is the cost/detail dial: 12 communities at level 1 is 13 model calls.

"What dominates extraction quality?"

The entity type list. Leaving it open-ended produced 340 distinct types where five were intended (CUSTOMER, CLIENT, ACCOUNT, ORGANISATION and COMPANY all meaning the same thing), the graph fragmented at type boundaries, and multi-hop accuracy was 52 percent instead of 79. Closing the list is one line of prompt and it was the largest single quality lever in the build. Also require descriptions rather than bare names, since descriptions are what make resolution and summarisation possible.

"What does it cost to index?"

Roughly two orders of magnitude more than vector RAG. Every chunk gets an extraction call, usually plus a gleaning round, then resolution adjudications, then community summarisation: about 14,000 LLM calls for 10,000 documents, a few hundred dollars and several hours, against a few dollars and a few minutes for embeddings. That cost is the central fact of the technique, and without incremental update it recurs on every refresh.

"How do you keep it fresh?"

Incremental update: extract only new chunks, resolve them against the existing graph rather than re-resolving everything, merge, and re-summarise only the affected communities. The awkward part is community structure, because new entities can change cluster membership, so you need a modularity-drift measure to decide when a full re-detection is due. In one case that took the daily cost from $310 to $6.20, which is the difference between viable and not.

Common misconceptions

"GraphRAG replaces vector RAG." It is comparable on simple lookup, which is most queries, and two orders of magnitude more expensive to index. The right architecture routes by question type and keeps vector RAG for the common case.

"A graph database makes it GraphRAG." Storing embeddings in Neo4j is vector RAG with a different store. The defining property is retrieval that follows relationships extracted from the text.

"Extraction is the hard part." Extraction is a prompt. Resolution is the hard part, and under-resolved entities mean a graph whose traversals silently find a fraction of what is known, with no error.

"The index builds once." For a corpus that changes, it rebuilds, and a full rebuild nightly is usually unaffordable. Incremental update with periodic community re-detection is what makes it a system rather than a demo.

"Global search is just summarisation." It is map-reduce over pre-computed community summaries, which is why it can cover a whole corpus. Asking a model to summarise ten retrieved documents is the thing global search exists to replace.

Interview delivery note

Say this verbatim: "GraphRAG addresses two question shapes vector RAG cannot: multi-hop, where the join is between documents, and global, where the answer is a property of the corpus. The failure to emphasise is global: asked for the main themes across four thousand tickets, vector RAG retrieves ten and summarises them, which is wrong in a way that looks right. And it costs about two orders of magnitude more to index, so I route by question type rather than replacing anything." The capability, the specific failure it fixes, and the cost that forces routing.

The senior-versus-staff separator is the entity type list. A senior engineer describes the pipeline correctly. A staff engineer knows that leaving entity types open-ended produces hundreds of near-duplicate types, that traversals die at type boundaries, and that closing the list moved multi-hop accuracy from 52 to 79 percent for one line of prompt. Knowing which knob dominates is the difference between having read the paper and having built it.

The second signal is incremental update. Saying "a full nightly rebuild was $310 a night, so the real engineering is resolving new entities against the existing graph and re-summarising only affected communities, with a modularity-drift trigger for full re-detection" shows you have thought about GraphRAG as a system that has to stay fresh rather than as an indexing run.

The Staff & Team Lead Interview Reference