Monitor-Guided Decoding: static analysis inside the sampling loop
TL;DR. Chapter 37 showed constrained decoding masking
the logits to a static grammar; Chapter 38
showed a language server computing the members that actually exist on a type.
Monitor-Guided Decoding (MGD) wires the two together: as a code model
generates, a monitor watches for a member access (the . in account.),
pauses decoding, asks the language server for the valid members of the
receiver's type, and masks the model's logits so it can only emit a token that is
a prefix of a real member. Hallucinated method names become literally
unsamplable. The hard part is that a valid identifier spans several model tokens,
so the mask must be maintained across steps as a prefix automaton over a trie
of legal completions, and this chapter builds exactly that in NumPy: a toy model
that hallucinates 66% of the time unconstrained is driven to 100% type-correct
by the monitor, while its preferences among the real members are preserved. Then
it reports the actual paper's numbers (compilation rate up ~19 to 25%, a 1.1B
model plus MGD beating 175B text-davinci-003), and closes the loop with Claude
Code: the API does not expose logits, so a black-box agent cannot mask tokens, but
it gets the same guarantee one level up, by grounding every symbol in the
language server and letting the compiler reject the hallucinations MGD would have
forbidden.
Contents
- The gap constrained decoding cannot close
- The monitor: a mask computed from the language server
- Multi-token identifiers: the prefix automaton
- The demo
- The formalism, in four equations
- The evidence: DotPrompts and the small-model result
- Where this lives, and how it reaches Claude Code
- Further reading
- Takeaways
This is where the last two chapters meet. Chapter 37
ended on constrained decoding: mask the logits so the model can only emit tokens
that keep the output on a legal path through a grammar. But it flagged a limit.
The grammar has to be written down in advance, and the single most useful
constraint in code generation cannot be: the set of methods that are valid after
account. depends on the type of account, which depends on imports, class
hierarchies, and libraries scattered across the whole repository. Chapter
38 built the tool that answers that question
live, the language server, and showed how a coding agent uses it to retrieve
the right symbols. Monitor-Guided Decoding takes the same language-server answer
and pushes it one layer deeper, into the sampling loop itself, so the model
cannot even emit a symbol that does not exist.
The gap constrained decoding cannot close
Start with the failure MGD was built to fix, taken straight from the paper's
opening example. A model is completing Java code that should build a
ServerNode. The correct continuation, .withIp(arr[0]).withPort(...).build(),
uses methods defined on ServerNode.Builder, a type declared in a different
file. Both a 175-billion-parameter model (text-davinci-003) and a small
open model (SantaCoder) instead write .host(arr[0]).port(...). It reads
perfectly. host and port are exactly the words a human would guess. They are
also hallucinations: ServerNode.Builder has no host and no port, so the
code fails to compile with "symbol not found". The model wrote plausible English,
not valid code, because the real member names live in a file it was not looking
at.
This is not a knowledge problem you can prompt your way out of. You could paste
the whole ServerNode.Builder definition into the context (the retrieval move
from Chapter 5), and it would help, but the model
can still ignore it and write host anyway, because nothing forces the output
to respect the type. Constrained decoding from Chapter
37 forces the output to respect a grammar, but a JSON
schema cannot encode "the members of whatever type arr resolves to at this
line". The constraint you need is real, it is checkable, and it is different at
every dereference in the file. It has to be computed on the spot.
Remember. The hallucination MGD targets is specifically the out-of-file identifier: a method, field, or type that is valid in the repository but not visible in the local context the model conditions on. Retrieval makes it more likely the model picks the right one; MGD makes it impossible to pick a wrong one. Those are different guarantees, and the paper shows they stack.
The monitor: a mask computed from the language server
MGD's answer is to run a second process, a monitor, in lockstep with the
decoder. Most of the time the monitor is asleep and decoding is exactly the
model's own sampling, so there is no overhead and no change in behaviour. The
monitor wakes on a trigger: a syntactic pattern in the tokens generated so
far that signals a constraint is about to apply. For the type-correct-dereference
monitor, the trigger is simple: the partial code ends with an object dereference,
some expression followed by a .. The moment the model emits that ., the
monitor fires.
When it fires, the monitor runs a static analysis over the repository. In
practice this is a language-server completion request, the request_completions
call from Chapter 38, issued at the cursor
position. The language server has already indexed the project, so it resolves the
type of the receiver (using imports, the class hierarchy, and any build-time
generated code) and returns the set of identifiers legal at that point: exactly
the members that type actually has. That set is the constraint. Everything the
model might want to write that is not in the set is a hallucination the monitor
is about to forbid.
Forbidding it is the logit mask from Chapter 37, applied verbatim. The monitor builds a mask $m$ over the vocabulary: allowed tokens get $m=1$, everything else $m=0$. Then it combines the mask with the model's logits $\ell$ using an operator the paper writes $\oplus$: where the mask is zero, the logit is reset to a large negative value $-K$; where the mask is one, the logit is left alone. Push $\ell \oplus m$ through softmax and the disallowed tokens get a probability of essentially zero, while the allowed tokens keep their relative preferences, renormalized to sum to one. This is the key property, and it is worth saying slowly: the monitor removes the wrong answers without choosing the answer. Among the legal members, the model still expresses its own judgment about which one fits. The language server decides what is possible; the model decides what is good.
Multi-token identifiers: the prefix automaton
If every valid member were a single vocabulary token, the story would end there:
one mask, one draw, done. It does not, and the reason is the tokenizer. A model's
vocabulary is made of sub-word pieces, so a member like withPort might be two
tokens, with and Port, and disconnect might be three. The language server
hands back whole identifiers as strings; the model emits sub-word tokens. The
monitor has to bridge that gap, and how it does so is the technically interesting
part of MGD.
The bridge is a prefix constraint maintained across steps. Think of the legal
member names as paths in a trie. At the first step after the ., the monitor
allows any token that is a prefix of some legal member. Then it watches which
token the model actually emits and prunes the trie: it drops every member the
emitted token does not begin, and it shortens the survivors by the piece just
written. The next step allows any token that continues one of the remaining
members, and so on. The constraint tightens with each token until an identifier is
complete, at which point the monitor sees an end marker (the ( of a call, a ,,
whitespace) and reverts to sleep, and decoding continues normally.
The subtle case, and the one worth engineering the demo around, is a shared
prefix. Suppose the type has a real member sendBatch and the model is also
tempted by a non-member sendEmail. Both start with the token send. At the
first step the monitor cannot rule either out, so it allows send. The
distinction only becomes decidable at the second step: after send, the trie of
legal completions permits Batch (finishing sendBatch) and the end marker
(finishing the bare send), but not Email, because no legal member is
sendEmail. So the mask at step two removes Email while keeping Batch. That
is the automaton doing real work across steps: the constraint could not be applied
at the first token and had to be carried forward. Any honest implementation of MGD
must handle this, and the demo below does.
Don't be confused. The monitor is not a filter that generates a full candidate and rejects it if wrong. It is a per-step mask that makes the wrong continuation impossible to draw in the first place, so no compute is spent exploring a doomed path and no backtracking is needed. It is the same discipline as truncation from Chapter 37, where a top-k token appeared 0.0% of the time across 4000 draws, except the allowed set is recomputed at every step from a trie of repository-valid identifiers instead of from a rank cutoff.
The demo
The script below builds the whole mechanism on a toy sub-word vocabulary. The
receiver has type Mailer with real members send, sendBatch, connect,
close; the model (a hand-built stand-in) is deliberately fond of members that do
not exist, sendEmail, sendAll, disconnect, so unconstrained it hallucinates
most of the time. The valid member sendBatch and the hallucination sendEmail
share the sub-word send, so the monitor is forced to act across two steps, not
one. allowed_next is the prefix automaton (which tokens keep us on a legal path
given what we have emitted); decode masks the logits to that set before
sampling. Then a seeded 3000-trial experiment measures the type-correct rate with
the monitor off and on.
"""Monitor-Guided Decoding from scratch: a language server steering the logits.
Monitor-Guided Decoding (Agrawal et al., NeurIPS 2023, from Microsoft's
multilspy work) closes the loop between the two previous labs. A code model
generates token by token; when it emits a member access like `mailer.`, a
*monitor* pauses decoding, asks a language server for the members that actually
exist on the receiver's type, and masks the logits so the model can only emit a
token that is a prefix of some valid member. Hallucinated method names become
literally unsamplable.
This lab builds that mechanism on a toy sub-word vocabulary so the interesting
part, the multi-step prefix constraint, is visible: the valid member `sendBatch`
and the hallucination `sendEmail` share the first sub-word `send`, so the
monitor cannot decide at the first token. It must keep constraining across
steps, exactly like the real prefix automaton. Then it measures the payoff over
3000 seeded trials: the model's type-correct ("compile") rate jumps from ~33%
to 100% with the monitor on, while its preferences among the *valid* members are
preserved.
numpy only. Deterministic (seeded).
"""
import numpy as np
END = "END" # the model has finished the identifier and moves on to '('
# The "language server" answer. For a receiver of type Mailer, these are the
# real members. Each is a sub-word token sequence ending in END. Real MGD gets
# this set from an LSP completion request; here it is the ground truth.
VALID = {
("send", END), # send
("send", "Batch", END), # sendBatch (shares 'send'!)
("conn", "ect", END), # connect
("clos", "e", END), # close
}
VALID_NAMES = {"".join(seq[:-1]) for seq in VALID} # {'send','sendBatch',...}
# The code model, as a stand-in: conditional next-token logits given the tokens
# emitted so far. It is deliberately fond of members that do NOT exist:
# 'sendEmail' (send+Email) and 'sendAll' (send+All) and 'disconnect' (dis+...).
# Unconstrained, it hallucinates most of the time.
MODEL = {
(): {"send": 3.0, "dis": 1.5, "conn": 1.0, "clos": 0.5},
("send",): {"Email": 2.5, END: 1.0, "Batch": 0.8, "All": 0.5},
("send", "Batch"): {END: 5.0},
("send", "Email"): {END: 5.0}, # only reachable without the monitor
("send", "All"): {END: 5.0}, # only reachable without the monitor
("dis",): {"conn": 5.0},
("dis", "conn"): {"ect": 5.0},
("dis", "conn", "ect"): {END: 5.0},
("conn",): {"ect": 5.0},
("conn", "ect"): {END: 5.0},
("clos",): {"e": 5.0},
("clos", "e"): {END: 5.0},
}
def softmax_over(tokens, logits):
z = np.array(logits) - max(logits)
e = np.exp(z)
return tokens, e / e.sum()
def allowed_next(prefix):
"""The monitor's constraint, computed live from the valid-member set: given
the sub-words emitted so far, which next tokens keep us on a prefix of some
real member? This is one step of the prefix automaton. END appears here
exactly when `prefix` already spells a complete member."""
prefix = tuple(prefix)
nxt = set()
for seq in VALID:
if seq[:len(prefix)] == prefix and len(prefix) < len(seq):
nxt.add(seq[len(prefix)])
return nxt
def decode(rng, monitor):
"""Decode one identifier after the `.` trigger. With monitor=True the
logits are masked to allowed_next(prefix) before sampling; with monitor=
False the model runs free. Returns the surface name it produced."""
prefix = []
while True:
table = MODEL[tuple(prefix)]
tokens = list(table)
logits = [table[t] for t in tokens]
if monitor:
ok = allowed_next(prefix)
keep = [(t, l) for t, l in zip(tokens, logits) if t in ok]
tokens, logits = [t for t, _ in keep], [l for _, l in keep]
toks, probs = softmax_over(tokens, logits)
choice = toks[rng.choice(len(toks), p=probs)]
if choice == END:
return "".join(prefix)
prefix.append(choice)
def trace(monitor):
"""Greedy single trace (argmax at each step) to show the mechanism."""
prefix, steps = [], []
while True:
table = MODEL[tuple(prefix)]
tokens = list(table)
logits = [table[t] for t in tokens]
banned = []
if monitor:
ok = allowed_next(prefix)
banned = [t for t in tokens if t not in ok]
kept = [(t, l) for t, l in zip(tokens, logits) if t in ok]
tokens, logits = [t for t, _ in kept], [l for _, l in kept]
pick = tokens[int(np.argmax(logits))]
note = ""
if banned:
note = f" [monitor masked: {', '.join(banned)}]"
steps.append((pick, note))
if pick == END:
break
prefix.append(pick)
return "".join(s for s, _ in steps if s != END), steps
def demo_traces():
print("=== The language server says Mailer has: "
+ ", ".join(sorted(VALID_NAMES)) + " ===\n")
print("=== Greedy trace WITHOUT the monitor ===")
name, steps = trace(monitor=False)
for tok, note in steps:
print(f" emit {tok:6}{note}")
verdict = "type-correct" if name in VALID_NAMES else "HALLUCINATION (no such member)"
print(f" -> mailer.{name}() [{verdict}]\n")
print("=== Greedy trace WITH the monitor ===")
name, steps = trace(monitor=True)
for tok, note in steps:
print(f" emit {tok:6}{note}")
verdict = "type-correct" if name in VALID_NAMES else "HALLUCINATION"
print(f" -> mailer.{name}() [{verdict}]")
print(" (note the mask fired at the 'send' branch: 'Email' and 'All'")
print(" were removed because sendEmail / sendAll are not members)\n")
def demo_rates():
print("=== 3000 seeded trials: hallucination rate off vs on ===")
N = 3000
for label, monitor in (("monitor OFF", False), ("monitor ON", True)):
rng = np.random.default_rng(0)
counts = {}
for _ in range(N):
name = decode(rng, monitor)
counts[name] = counts.get(name, 0) + 1
valid = sum(c for n, c in counts.items() if n in VALID_NAMES)
print(f"\n {label}: type-correct (compiles) = {valid / N:.0%}")
for name in sorted(counts, key=lambda k: -counts[k]):
tag = "" if name in VALID_NAMES else " <- hallucination"
print(f" mailer.{name+'()':14} {counts[name] / N:5.1%}{tag}")
print("""
Monitor off: the model's own preferences send it to sendEmail, sendAll,
and disconnect the majority of the time, so only ~1 in 3 completions
would compile. Monitor on: the hallucinations are unsamplable, so every
completion is type-correct, yet the model still chooses freely AMONG the
real members ('send' beats 'connect' because the model preferred it). MGD
removes the wrong answers without picking the answer, which is why a small
model plus a monitor can match a much larger unconstrained one on
type-correctness.""")
if __name__ == "__main__":
demo_traces()
demo_rates()
Running it:
=== The language server says Mailer has: close, connect, send, sendBatch ===
=== Greedy trace WITHOUT the monitor ===
emit send
emit Email
emit END
-> mailer.sendEmail() [HALLUCINATION (no such member)]
=== Greedy trace WITH the monitor ===
emit send [monitor masked: dis]
emit END [monitor masked: Email, All]
-> mailer.send() [type-correct]
(note the mask fired at the 'send' branch: 'Email' and 'All'
were removed because sendEmail / sendAll are not members)
=== 3000 seeded trials: hallucination rate off vs on ===
monitor OFF: type-correct (compiles) = 34%
mailer.sendEmail() 44.2% <- hallucination
mailer.disconnect() 16.3% <- hallucination
mailer.send() 10.3%
mailer.sendBatch() 9.0%
mailer.connect() 8.8%
mailer.sendAll() 6.0% <- hallucination
mailer.close() 5.5%
monitor ON: type-correct (compiles) = 100%
mailer.send() 44.7%
mailer.sendBatch() 37.8%
mailer.connect() 11.0%
mailer.close() 6.5%
Read the two traces first. Without the monitor, greedy decoding writes
mailer.sendEmail(), a clean hallucination: sendEmail is not a member, so it
would not compile. With the monitor, the mask fires twice. At the first token it
removes dis (no legal member begins that way, so disconnect is dead on
arrival), and at the second token, having emitted send, it removes Email and
All while allowing the identifier to end, producing the valid mailer.send().
That second mask is the shared-prefix case: the monitor could not forbid the
hallucination at the send step, only at the step after.
Now the numbers. Unconstrained, the model's own preferences carry it to
sendEmail, disconnect, and sendAll a clear majority of the time, so only
34% of its completions would compile. With the monitor on, the three
hallucinations are drawn 0.0% of the time, so every completion is
type-correct, a jump to 100%. And crucially the second block is not a single
forced answer: the monitor left the model free to prefer send over sendBatch
over connect over close, in the model's own order. It deleted the wrong
answers and let the model rank the right ones, which is exactly the property that
makes MGD safe to bolt onto any model without retraining it.
The formalism, in four equations
The paper states this precisely, and the four equations map one-to-one onto the
demo's decode loop. A monitor for a property $\varphi$ is a tuple $M_\varphi =
(A_\varphi, s_0, S, \texttt{pre}, \texttt{update}, \texttt{maskgen})$: a static
analysis $A_\varphi$, a wait state $s_0$, a set of states $S$, a trigger
pre, a state-transition update, and a mask generator maskgen. Running the
language model $L_\theta$ jointly with the monitor, written $L_\theta ,|,
M_\varphi$, the probability of the next token is
$$ (L_\theta | M_\varphi)(x_{n+1}) = \begin{cases} \texttt{softmax}(\ell)[x_{n+1}] & \text{if } s = s_0 \ \texttt{softmax}(\ell \oplus m)[x_{n+1}] & \text{otherwise} \end{cases} $$
with $\ell = L_\theta(\cdot \mid x_1 \dots x_n)$ the model's logits, $m = \texttt{maskgen}(s, V)$ the mask over vocabulary $V$, and the state advancing as
$$ s' = \begin{cases} A_\varphi(x_1 \dots x_n; C) & \text{if } s = s_0 \wedge \texttt{pre}(s; x_1 \dots x_n) \ \texttt{update}(s, x_{n+1}) & \text{otherwise.} \end{cases} $$
In the demo's terms: s = s_0 is "before the .", where sampling is the model's
own softmax (the monitor is invisible). pre firing is the . trigger.
$A_\varphi$ is the language-server completion call that returns the member set,
which becomes the state. maskgen is allowed_next, the set of tokens that keep
the prefix legal. The $\oplus$ operator is the -inf masking. And update is the
trie-pruning that runs after each emitted token until the identifier ends and the
monitor returns to $s_0$. The equations are the loop; the loop is the equations.
Two properties fall out of this shape for free. If $A_\varphi$ returns an empty
set (the analysis found nothing to constrain), the monitor abandons the attempt
rather than masking everything. And because a monitor is just an automaton over
the vocabulary, two monitors can be combined by taking the product of their
state spaces, which is how the paper guides "type-correct member and correct
number of arguments" at once.
The evidence: DotPrompts and the small-model result
The paper ("Monitor-Guided Decoding of Code LMs with Static Analysis of
Repository Context", Agrawal, Kanade, Goyal, Lahiri, and Rajamani, NeurIPS 2023)
measures this on real Java. They built two artifacts, both released: PragmaticCode,
100 open-source Java repositories with full build environments, deliberately
chosen from projects published after the models' training cutoff so the answers
could not have been memorized; and DotPrompts, 1,420 methods and 10,538
dereference prompts derived from them, where each task is to complete a method
starting from a . dereference. The headline metric is Compilation Rate (CR):
splice the generated method body back into the real repository, run a clean build,
score 1 if it compiles. Three supporting metrics check agreement with the ground
truth: Next Identifier Match, Identifier Sequence Match, and Prefix Match.
Decoding is nucleus sampling at top-p 0.95, six samples, reported as score@k.
The results are the reason MGD is worth a chapter. Across every model they tried, adding MGD lifts the compilation rate by roughly a fifth to a quarter:
| Model | CR without MGD | CR with MGD | relative gain |
|---|---|---|---|
| CodeGen-350M | 52.43 | 65.37 | +24.7% |
| CodeGen-2B | 57.01 | 70.91 | +24.4% |
| CodeGen-6B | 58.64 | 72.28 | +23.3% |
| SantaCoder-1.1B | 59.97 | 73.03 | +21.8% |
| text-davinci-003 (~175B) | 62.66 | 74.26 | +18.5% |
Read down the last column and then across the middle two. Every model gains, and
the gain is large. But the sharper result is the comparison between rows: SantaCoder,
a 1.1-billion-parameter model, with MGD reaches a 73.0 compilation rate, higher
than text-davinci-003, a roughly 175-billion-parameter model, without it (62.7).
A monitor built from a language server closed a hundred-fold size gap on
type-correctness. The paper reports the same pattern on next-identifier match
(SantaCoder plus MGD at 88.4 beats text-davinci-003 at 86.2) and shows the
effect is strongest exactly where you would predict: on long, repository-specific
identifiers that span many sub-word tokens, the very names a model is least likely
to spell correctly on its own and most likely to get from the language server.
MGD also stacks with the retrieval methods from Chapter
5: feeding the model better context and constraining
its output are complementary, and the best configuration in the paper combines
both.
Where this lives, and how it reaches Claude Code
The implementation is open source. The language-server bindings are
multilspy (Chapter 38 built a toy of the
same idea), and the monitors live in the monitors4codegen repository
alongside the paper's datasets. Beyond the dereference monitor, the paper
demonstrates the pattern generalizes: a monitor triggered on new for valid
class instantiation, one triggered on case to force valid enum constants, a
stack-based monitor that checks the correct number of call arguments, and monitors
driven by richer analyses like typestate ("you cannot read a file handle after
close") across Java, C#, and Rust. The trigger and the analysis change; the
mask-the-logits machinery does not.
Now the honest limitation, and it is the one that connects straight back to
Chapter 37. MGD needs the model's logits. The whole
mechanism is a mask applied to $\ell$ before softmax, so it requires white-box
access to the decoder. The Anthropic API does not expose logits, does not accept a
logit_bias, and on the frontier models does not even accept temperature. You
cannot run MGD against Claude the way you run it against a local SantaCoder,
because the surface where the mask would attach is not there. The paper
acknowledges the same wall: for black-box models like text-davinci-003 it could
only approximate the masking.
So how does a black-box coding agent like Claude Code get MGD's guarantee that
generated symbols are real? It moves the constraint up one level, from the token
to the tool. It cannot mask the logits, so instead it (1) grounds every
symbol before writing: the language-server tools from Chapter
38, find_symbol, get_symbols_overview, and
find_referencing_symbols, only ever return identifiers that actually exist, so
when the agent reads the type's members before editing, the correct names are in
its context and the incorrect ones are conspicuously absent; and (2) checks
after writing: it runs the build or the type checker and feeds the diagnostics
back into the loop, so a hallucinated host() that MGD would have forbidden at
the token level is instead caught by the compiler and corrected on the next turn.
The two approaches bracket the same goal. MGD prevents the hallucination before
the token is emitted, with perfect precision and zero wasted output, but only with
white-box access. Tool-grounding plus diagnostics catches it after, costing a
build and a correction turn, but works on any model behind any API. The toy above
is the clean, white-box ideal; a Claude Code session that reads a symbol with the
language server and then lets the type checker reject what it got wrong is the same
constraint, enforced a layer higher, on a model whose logits you will never touch.
That is the through-line of this final part. Chapter 37 showed that generation is a stream of masked logits and that the frontier models hand you the grammar form of the mask (structured outputs) while withholding the raw dials. Chapter 38 showed the tool that turns a whole repository into a queryable set of facts. This chapter showed the tightest mask of the three, the one computed live from those facts, and showed why you run it directly when you own the decoder and approximate it with tools and compilers when you do not.
Further reading
- Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context (Agrawal, Kanade, Goyal, Lahiri, Rajamani, NeurIPS 2023, arXiv:2306.10763, also titled "Guiding Language Models of Code with Global Context using Monitors"). The source for everything in this chapter: the monitor formalism, the prefix masking, DotPrompts, and Table 1.
- microsoft/monitors4codegen (github.com). The paper's code and datasets: the dereference, instantiation, switch-over-enum, and argument-count monitors, plus the PragmaticCode / DotPrompts release. Read the monitor implementations against the toy here.
- microsoft/multilspy (github.com). The language-server bindings the monitor
queries, dissected in Chapter 38. The
request_completionscall is the $A_\varphi$ of this chapter. - Grammar-Constrained Decoding / Outlines / XGrammar. The static-grammar cousins of MGD from Chapter 37. MGD is what you get when the grammar is replaced by a live query to a static analyzer, so reading these side by side makes the generalization concrete.
Takeaways
- Constrained decoding masks logits to a grammar you write in advance; MGD masks them to a set computed live by a language server, so the constraint is the repository's real types, not a static schema.
- The monitor sleeps until a trigger (a
.dereference), then queries the language server for the legal members and masks every other token. It removes the wrong answers without choosing the answer, leaving the model free among the valid members. - Because a valid identifier spans several sub-word tokens, the mask is a
prefix automaton: allow tokens that continue a legal member, prune the trie
as each token is emitted, revert to sleep at the end marker. The demo's
shared-prefix case (
sendBatchvssendEmail) forces the constraint to act across steps. - The from-scratch demo took a model that hallucinated 66% of the time to 100% type-correct with the monitor on, preferences preserved. The paper's real result: compilation rate up ~19 to 25% across model sizes, and a 1.1B model plus MGD beating a ~175B model without it.
- MGD needs white-box logit access, which the Anthropic API does not give. A black-box agent like Claude Code reaches the same guarantee one level up: ground symbols through the language-server tools so only real names enter context, and check with the compiler so hallucinations are caught and corrected in the loop.
👉 That closes the loop from a single logit to a repository-wide constraint. Steering generation was the output side; the next part returns to the most personal input-side question left open: what your own durable facts cost, how the real memory systems carry them, and how to prove any of it with an eval harness. Continue to The static-facts ledger.