Fastindex: prune the folders that cannot help

**Fastindex** walks a file tree folder by folder. It prunes branches that cannot help, opens only the ones that might, and returns line spans. Prepare the tree once. Query it many times. Cheaper and Faster than rediscovering the corpus with naive grep on every agent turn.

<table_of_contents/>

The problem

Agents usually treat a wiki as a bag of chunks. Grep, BM25, or embeddings score every shard, take top-k, and hope the right page is in the pile.

That works when you already know the keyword. It fails when the wiki was written with structure: folders, index.md files, and frontmatter already say what lives where.

Flat search ignores that structure. It returns pages that share words but not the fact. The agent then spends another turn sorting noise from signal.

What Fastindex does

Fastindex is tree retrieval for hierarchical markdown wikis (OKF-shaped bundles). No embedding index. The filesystem is the catalog.

The walk:

1. Read the folder index and its children.
2. Ask which children can help.
3. Prune the rest unread.
4. Open only useful children.
5. Return line ranges that evidence the query. Do not generate an answer.

Two commands:

• fastindex prepare writes short routing index.md files bottom-up (reason once).
• fastindex query prunes, descends, and returns spans (act many times).

Two query strategies:

• tree-reason: an LLM chooses which children to open. Works. Costly per hop. Free-form JSON can fail closed.
• tree-decision: same walk, typed choice over children (plus none). Calibrated probabilities. No prose to parse.

I tried the same prune walk earlier with chat models. Latency and cost stacked at every hop, so it never beat grep. Classifier / System One models (Jev via classifier.dev or TypeSafe, and Laya) make each prune cheap enough to run.

Case study

On the sample wiki in the repo, a local question is answered by pruning sibling branches that share vocabulary but not the fact, descending one path, and returning the span on that leaf. Grep would have opened the false friends.

What comes back

Each hit is:

• path
• start line / end line
• span text

Evals rank on span recall first, then latency and call count. Budgets on time and model calls truncate to whatever spans were already found.

Why this beats naive grep

Grep matches words. It does not know which subtree owns a fact class.

Fastindex uses the tree as a prior:

• Folder names and short index blurbs for routing
• Concept frontmatter (title, description, when_to_use, tags) before reading bodies
• Correct prune = zero leaf I/O; missed prune = you pay for the whole subtree

On single-hop local questions against an organized tree, that beats BM25 / FTS in the repo harness because the prune removes the noise those methods return.

Out of scope: multi-hop via wikilinks. The walker only goes parent to child.

Gate cost

Ten sample queries, same routing indexes, top_k=2. Prep cost excluded. Single runs. Not a scale claim. Point: typed decisions make each prune cheap enough for the walk to win.

On this fixture:

• TypeSafe Jev matches the best LLM span quality at about 3× lower latency and 8× lower query cost than tree-reason
• classifier.dev Jev is a good default: ~2.4s, 0.90 recall, free under current terms

The comparison that matters is prune-and-descend vs score-everything, once the gate is cheap.

Where it loses

• Messy trees: nothing to prune; the walk becomes expensive grep
• Known keyword + one-shot breadth: grep is still faster
• Cross-page links: not supported yet
• Prep costs tokens once; only pays if you query more than once

What shipped

fastindex is a research preview:

• prepare: bottom-up routing indexes
• query: tree-reason or tree-decision
• adapters: classifier.dev (Jev / Laya), TypeSafe Jev
• lint / generate-index for OKF bundles
• evals/ against BM25, FTS, and optional external retrievers

uv sync --extra dev
uv run fastindex prepare PATH
uv run fastindex query PATH "your question" \
  --strategy tree-decision --decision-model classifier/jev

Close

A structured wiki already has a retrieval plan. Fastindex runs it: prepare once, prune what cannot help, return the lines that can. Against naive grep, you get narrower context on local questions when folders are honest. Classifier gates are why the walk is cheap enough today. Grep still wins for one-shot keywords. Multi-hop still needs link following.

Sources

• fastindex
• classifier.dev
• TypeSafe System One / Jev
• Google Cloud, OKF spec