Cortex-Guided Fuzzing: Letting an LLM Write the Grammar

The plateau problem

Coverage-guided fuzzers are brilliant at bit-flipping their way through binary formats and hopeless at structured text. Point one at the Cortex service's configuration parser — a nested, schema-validated policy language — and it spends days rediscovering that { must be balanced. Coverage climbs fast, then flatlines. Every mutation that matters is rejected by the parser's front-end validation before it can reach the interesting code behind it.

The usual fix is to hand-write a grammar. That works, but it is slow, it goes stale as the format evolves, and it encodes only the structures a human thought to write down. We tried something else: put an LLM inside the loop and let it write and revise the grammar in response to coverage.

Cortex is a fictional product used to illustrate the technique.

Architecture

The loop has four moving parts:

  corpus ──► fuzzer ──► coverage delta
     ▲                      │
     │                      ▼
  LLM grammar ◄──── frontier analysis
   synthesizer      (what's rejected? what's uncovered?)

The fuzzer runs normally. Periodically, a controller collects two things: inputs that were rejected by the parser front-end, and basic blocks that remain uncovered. It hands both to the LLM and asks for grammar rules that would produce inputs threading past the rejections toward the cold code.

Bootstrapping the grammar

We seed the LLM with a handful of valid samples and the parser's error messages, then ask for a grammar in a compact PEG-like DSL:

prompt:
  Here are 5 valid Cortex configs and 12 rejected mutations with
  their parser errors. Emit grammar rules (PEG) that generate
  configs which PASS validation and exercise nested policy blocks.
  Prefer structures NOT present in the samples.

The model responds with rules the fuzzer can drive directly:

config     <- "version:" INT policy+
policy     <- "policy" IDENT "{" rule* nested? "}"
nested     <- "delegate" "{" policy+ "}"    # recursion the corpus lacked
rule       <- action WS matcher
action     <- "allow" / "deny" / "audit"
matcher    <- "cidr(" IP "/" INT ")" / "tag(" STR ")"

The critical line is nested — the LLM inferred a legal recursive construct that none of our seed samples contained, and it turned out to reach an entire subsystem the fuzzer had never touched.

Closing the loop

Grammars go stale the moment the parser rejects what they generate, so we feed results back every cycle:

while budget_remaining():
    inputs = grammar.generate(n=5000)
    cov, rejects = fuzzer.run(inputs)
    frontier = analyze(cov.uncovered, rejects)
    if cov.plateaued():
        grammar = llm.revise(grammar, frontier, examples=rejects[:20])

When coverage plateaus, llm.revise gets the current grammar plus fresh evidence of what's still rejected and what's still cold, and returns a patched grammar. The LLM is not generating inputs — that would be slow and expensive. It generates the generator, and the cheap fuzzer produces millions of inputs from it.

Guardrails

An LLM in a loop will happily drift. Three constraints keep it honest:

Grammars must compile and validate. A returned grammar that fails to parse, or whose sample outputs are all rejected by Cortex, is discarded and the previous grammar is kept.
Reward only new coverage. We accept a revision only if it strictly increases edges hit; otherwise we roll back. The model cannot make things worse, only better or neutral.
The LLM never sees crashes as a goal. It optimizes coverage. Crash triage is a separate, deterministic stage — we don't want the model reward-hacking toward shallow, uninteresting faults.

Results

Against the Cortex config parser, over a fixed 72-hour budget:

Baseline coverage-guided fuzzing: ~54% edge coverage, plateaued at ~18 hours.
Static hand-written grammar: ~67%, plateaued at ~40 hours.
Cortex-guided (LLM in loop): ~81%, still climbing at cutoff.
Three distinct crashes found, all in the delegate/nested path the LLM discovered — including one recursion-depth issue that corrupted parser state.

The headline is not the raw coverage number; it is where the coverage went. The LLM's value was inferring legal-but-unusual structures a human grammar-author would have needed the spec (and a lot of patience) to enumerate.

Practical notes

Keep the LLM out of the hot path. One grammar revision per plateau, thousands of fuzzer-generated inputs per revision. The economics only work if the model generates generators, not test cases.
Feed it errors, not just successes. Rejected inputs with their parser diagnostics are the most information-dense signal you can give it.
Version your grammars and coverage together so you can always roll back a bad revision.

Structured-input fuzzing has been stuck on the same wall for years: you need to speak the format to get past the front door. It turns out an LLM is very good at learning to speak a format from examples and error messages — which is exactly the key that fuzzer was missing.

#fuzzing#ai#llm#harness#coverage