Sandarbha: a code-intelligence engine that refuses to guess

The problem

A decades-old billing platform is not one codebase. It is four, written in different languages by different generations of engineers: a declarative data dictionary, a data layer of embedded-SQL C (Pro*C), batch modules machine-translated from COBOL to C, and the plain C underneath all of it. The questions engineers ask every day cut across all four:

• What columns does this table have, and which files declare them?
• What writes this column, in which statements, in which services?
• Which batch modules reach this service?
• If I change this, what has to be retested?

The answers lived in a few senior engineers’ heads and in grep. Pointing an AI assistant at the tree does not fix that. It makes wrong answers faster and more confident, and on a billing system “plausible” is not a safe property.

Sandarbha (Sanskrit for “context” or “reference”) is the tool I designed to answer those questions from evidence. It runs on-premises, reads the source mechanically, and serves one engine through a CLI, an HTTP API with a web UI, and an MCP server, so engineers and AI agents ask the same thing instead of guessing.

One promise

Everything else is the work of keeping that sentence true under pressure:

• When two source files disagree about a fact, it reports the disagreement and refuses to pick a winner.
• When it could not read some files, it says so and marks the affected counts as lower bounds.
• When a snapshot is too old to answer a kind of question, it says that instead of answering with an empty list.

An empty result and “I could not tell” are different sentences. The product never substitutes one for the other.

Architecture in one picture

The short version: sources are read mechanically into facts. Facts are the only authority. Everything else is a projection that can be thrown away and rebuilt. Readers only ever see a complete, immutable snapshot.

It is a Rust workspace of thirteen crates plus a SolidJS front end. The dependency direction is enforced by a test that parses the crate manifests, and a second assertion checks the converse so that the test cannot pass vacuously. Apart from one allocator call, unsafe Rust is confined to the compiler tier.

Four tiers of evidence

• tier 1 Dictionary Tables, columns, domains, keys and indexes, read through the dictionary's own grammar rather than inferred from samples. An early sampling-based reader produced a plausible count that was silently short. Reading the real grammar recovered what it had missed.
• tier 2 Data layer Embedded-SQL services. Each statement is isolated, macro-expanded and parsed against a SQL subset. Whether a WHERE clause covers a primary key is a dictionary question, so this reader wraps the dictionary reader instead of sitting beside it.
• tier 3 Batch Machine-translated COBOL-to-C modules. The translator's macro vocabulary is turned back into sections, calls, domains and variables, and every fact records which reader supplied it, so an answer can say how it knows.
• tier 4 Compiler C through a statically linked libclang, under the include paths the real build used, reconstructed from its make fragments (inferred, and flagged as inferred, where none exists). It derives control-flow graphs, control and data dependence, an interprocedural dependence layer, cross-unit call links and Tuxedo service bindings. Where clang's error recovery may have dropped code, dependences are reported as lower bounds.

Sealed snapshots, and three ways a sync can go

A sync reads the corpus, builds a candidate, validates it against more than thirty named checks, and only then seals it. A sealed snapshot is immutable. A reader resolves the active snapshot once and sees a world that cannot change underneath it, and a candidate that fails validation never replaces the active one.

A sync also decides which of three paths it is on before it reads anything expensive:

• path 1 Nothing changed Stop after discovery. The active snapshot stays exactly as it was.
• path 2 Representation only Line endings or whitespace moved but meaning did not. Carry the facts forward without re-reading.
• path 3 Meaning may have changed Run the full extraction, validate, seal.

Content identity is recorded twice, as raw bytes and as canonical logical bytes, which is what makes the second path possible. The ingestion recipe is a third input, so changing how files are read defeats the “nothing changed” shortcut instead of quietly serving stale facts. Reader installs, including a Windows build without the C front end, pull a sealed snapshot and verify its length and BLAKE3 digest before publishing it.

The honesty vocabulary

“No rows” has at least six meanings in this product, and a careless interface renders all six the same way. An engineer who reads “nothing writes this column” and stops looking has been misled by an answer that was, at the level of rows returned, perfectly correct.

• 1 I read it and found nothing An empty list. A real measurement.
• 2 I did not read it null, never an empty list.
• 3 The list was cut short truncated, together with the total.
• 4 I cannot say whether it was cut truncated is null, and the sections without a total are named.
• 5 This snapshot cannot answer that kind of question capability_unavailable_in_snapshot. It says nothing about whether the thing exists.
• 6 It genuinely is not there not_found. The only one of the six that is evidence of absence.

$ sandarbha column LEDGER_ENTRY.POSTEDnot_found  no such column here $ sandarbha find "post ledger entry"tiers 1-6  3 matchesnames:  capability_unavailable_in_snapshot  sealed before name search existed;  says nothing about whether a name  exists. $ sandarbha writers LEDGER_ENTRY.STATEwriters        0unread_writes  2   lower bound→ “nothing readable writes this”,  not “nothing writes this”

The type system carries the distinction. Where a collection can fail to be derived at all, its type is Option<Vec<T>>, not Vec<T>, so “asked and found nothing” and “never asked” cannot be confused by a caller. Failure kinds are a closed, stable set of tokens defined once, so the CLI’s JSON, the MCP server’s errors and the HTTP API cannot grow different vocabularies for the same failure. A test asserts that the “capability unavailable” message names its remedy while “not found” does not, because two messages that differ only by a token read identically to a tired human.

When sources disagree

Two dictionary files can declare the same field differently. Most tools pick one, usually whichever was read last. Sandarbha keeps both claims, cites the file and content hash behind each, and reports the conflict. It never resolves it, because choosing silently is how a wrong answer acquires a confident voice.

$ sandarbha conflictsLEDGER_ENTRY.STATUS  A  length 8   ledger_a.def  7c1e…  B  length 12  ledger_b.def  02d9…  resolved by nobody (both kept)

Order independence is a property of a data structure here, not a discipline. Observations are grouped by canonical identity in an ordered map, so by the time a group is resolved the order its members arrived in no longer exists. Careful people still write file-order bugs. The structure makes them hard to write.

Determinism: the same corpus, the same bytes

The same corpus must produce a byte-identical snapshot regardless of thread count, filesystem order, or how many times it is run. This outranks every other property. A change that depends on hash-map iteration order, parallel completion order, a wall clock or a process id is wrong even when every test passes.

Reproducible is narrower than correct. Nothing here proves the facts are right. Determinism means there is no second answer for the same input, which is what makes a wrong answer findable at all.

• vary Arrival order Shuffle the order inputs arrive in. The snapshot must not change.
• vary Thread count Run with different worker pools, including a pool of one.
• repeat Run it again Same corpus, same bytes, run after run.
• hardest Hash seeds Rust seeds its hash maps once per process. A loop inside one test varies the pool width and never the seed, so a value that depended on hash order would pass a thousand repeats and differ on the next process start.

So the proof re-executes the product as separate processes, and it repeats a pool width on purpose: a difference between two runs at the same width can only come from the seed. One trap is worth passing on. Each run gets its own data directory. Reuse one, and every run after the first takes the “nothing changed” path, extracts nothing and reports an identical digest, which reads exactly like agreement and is the absence of a measurement.

One service, four surfaces

A CLI with 60 subcommands, an HTTP API with 53 routes serving a SolidJS UI, and an MCP server with 51 tools all call the same query service. None of them can reach storage, and none contains query logic. Contract tests compare the CLI’s JSON, the MCP results and the HTTP payloads after transport envelopes are normalised, and the browser’s TypeScript models are generated from the Rust report types, so there is one report schema instead of a hand-written copy that can drift. Every answer cites its source file and content hash and carries its own limits.

Making the MCP server cheaper to meet

A model that connects to 51 tools pays for all their descriptions before it asks anything. From release 0.3.0-rc.10 the server offers a five-tool entry-point lens by default: find, explain a function, read a name, path conditions and status.

A call to a tool outside the lens is refused with a message that names the flag. Every answer counts the follow-up pointers it had to withhold, so it never points at a tool the session cannot call and never hides that it was cut. The lens is the full router with some routes removed, so a tool on both surfaces is one handler and cannot answer differently on each.

Proving that the tests prove something

The testing rule is blunt: a test that can pass without asking the intended question is defective. Earlier research on the legacy system turned up tests that passed reliably and proved nothing. A vacuous test is worse than no test. No test is honest about missing coverage, while a vacuous one asserts coverage that does not exist, and every later decision is made on that false basis.

The enforcement is mutation. For each required test, deliberately invert the behaviour it claims to guard, confirm that it fails, revert, and confirm that it passes again.

• mutation File order must not decide a fact Return the first grouped value instead of reporting a contradiction. Four tests fail.
• mutation A no-change sync runs no parser Force the full path when nothing differs. Its own guards fail; the guards for the other paths stay green.
• mutation A new recipe defeats the shortcut Drop the fingerprint comparison. Its own guard fails and the others stay green.

The most useful mutation was the one that caught nothing. A status summary used to count artifacts by loading them all and grouping in Rust. It was rewritten to group in the database, which left two code paths over one truth, and the loser drifts silently: a summary that has stopped agreeing with what it summarises still renders perfectly. Dropping one artifact kind from the new query failed no test at all, so a test now checks the two views against each other.

The first attempt at that mutation filtered on a kind that does not exist. The query returned the same rows and the suite stayed green. That is the failure mode of the whole exercise: a green suite under a mutation is only evidence when the mutation changed an observable answer.

More than 2,800 automated tests (about 2,200 in Rust and 600 in the UI) gate every release.

Shipping to servers that are not mine

The tool has to run where the code lives, so delivery is part of the design:

• Nothing leaves the machine. On-premises, no telemetry, no outbound network calls.
• A standalone binary for each target. A glibc 2.28 build for RHEL 8 and 9, a fully static musl build for RHEL 7 that needs nothing installed, and a Windows reader that pulls a snapshot a Linux host sealed.
• Verified before unpacking. A fetch script checks SHA-256 checksums and refuses to unpack anything it could not verify.
• No installer, no root. GitLab CI/CD builds and publishes every release.

The release notes carry a Known limitations section, because a product that derives knowledge must be explicit about what it does not know, and a release that hides its limits is committing the same sin as an answer that does.

The changes were unglamorous: bounding the validation read-back, narrowing the compiler tier’s unit cache, handing freed heap back to the kernel, limiting the reads in flight, storing eleven tables without a rowid, and logging resident memory at every phase so that the next regression can be found.

Building it with AI agents, and what went wrong

Sandarbha was built in Cursor with AI coding agents. That is part of the story, not a footnote to it.

• Rules and skills carry the contract. Always-on rules state the guarantees that must not be traded, the real grammars decide what a source file means, and a web rule insists that empty and unavailable look different and that fixtures are labelled as fixtures.
• Parallel agents, one discipline. Later releases ran as many short-lived agent sessions, each on its own branch (the history holds well over a hundred), under a written release discipline: branch, commit as you go, walk the docs, tag last.
• Pace. Over 1,800 commits and more than twenty tagged releases in about five weeks of repository history.

The project’s own retrospective is blunt about the failure mode. The agents invented answers too. Not by writing fictional SQL, but by shipping a layer that was computed, tagging it ready, and leaving the next person to discover that it was unreachable, unreadable or confidently wrong. That happened four times. Computed, usable and true are three different things, and the response was rules, tests and the honesty vocabulary above. One release candidate was demoted when real use showed it was not ready, and a user interface that rendered a network timeout as “nothing is named that” was fixed to say what it did not know.

What I took from it

• Make the type system carry honesty. If “never asked” and “found nothing” have different types, nobody has to remember the difference.
• Put determinism in the data structure, then test it where ordinary tests cannot reach, in separate processes.
• Break the code to check the tests. Reasoning about whether a test is meaningful is not a substitute for actually breaking the thing it guards.
• Mutation testing proves only the tests you have. It says nothing about code they never touch. The bugs it missed were in the reporting layer, where no test looked.
• Ship the limits with the release. An answer that hides what it does not know is wrong, and so is a release.

Related: An AI knowledge layer for legacy code, a different approach to the same family of problems.