Slides: How Neural DB works (PDF)

Life of a query: NeuralKG2 (Neural DB)

Design document, revised 2026-10-30. Covers NeuralKG2 main (20ab092) as deployed in revision neuralkg2-00174-dec. Every claim carries a file and line reference; the measured figures in sections 3.5 and 4 were taken from the running service on 2026-10-09. Sibling documents: ARD (the finder NeuralKG2 calls), QDFeeds (QDFeeds), NLWeb samples (the NLWeb samples).

0. How to read this

NeuralKG2 (the Neural DB, at kg.neuralweb.dev) answers a question in plain words with an answer whose every number comes from a published source. You give it "What is India's GDP?" or "Which US states border Canada?"; it returns claims, each pointing at the receipt it rests on, disclosures about what was assumed, charts, and a record of how the answer was made (stages, models, cost). The same pipeline serves the Neural DB page (also restricted to one source with ?sources=datacommons), a JSON /ask, and an MCP tool.

Words used throughout

• Reading: one structured interpretation of a question (a typed query such as RankingQuery). A question may have several.
• Slot: one piece of data a reading needs (a value for one entity, a quantity for every member of a population, the members themselves, a time series, a published fact, or a breakdown into categories).
• ARD entry / document: one table document describing one published variable from one source (see the ARD document).
• Plugin: the code that knows one data source: how to choose among its documents, resolve a place or company, build a request, read the response.
• Receipt: a number returned by a source with its unit, period, definition and provenance. Nothing is computed from anything that is not a receipt.
• Annotated query (aq): the reading plus everything each stage learned about each slot, snapshotted after every stage.

1. Where NeuralKG2 sits

flowchart LR
  U[Reader or client] -->|question| KG[NeuralKG2<br/>kg.neuralweb.dev]
  KG -->|describe the quantity| ARD[ARD finder]
  KG -->|chosen request| P[Source plugins]
  P --> DC[Data Commons API]
  P --> WB[World Bank, Census, SEC, ...]
  P --> OD[Open-data publishers<br/>Eurostat, OECD, PxWeb, SingStat, ...]
  KG <-->|bounded judgments| OR[(OpenRouter models)]

One Python HTTP server (neuralkg2/space.py) on Cloud Run (neuralkg2, us-central1). Nothing is stored between questions except in-memory caches and the per-run traces, which are also copied to Cloud Storage (gs://neuralweb-neuralkg2-traces/runs/<run_id>/, trace_archive.py) after the answer is sent. ard-gcp.json maps 25 ARD source directories to plugins: Data Commons, World Bank, UN SDG, Census (ACS, business statistics), SEC, nonprofits, Treasury, CDC PLACES, NIH, NSF, USAspending, and twelve open-data publishers that share one table core (plugins/cubes.py: Eurostat, OECD, Census trade, seven PxWeb statistics offices, SingStat, Philippine Statistics Authority; see docs/open-data-sources.md).

2. Rules and models

1. A model judges; code computes. Models choose among bounded options, rank or classify. Arithmetic, units, periods, joins and status are code. Any model call during the execute phase raises (runtime.py:200-201); plugin methods that resolve meaning are blocked there too (plugins/base.py:36-43).
2. Everything a model returns is validated: choices must be one of the offered ids, JSON must parse, arithmetic is rechecked. A bad reply is retried once or becomes a visible failure.
3. Failure is visible. Every stage failure becomes an annotation shown to the reader, and a missing part downgrades complete to partial.
4. Cost and time are accounted per stage. Seconds, model calls, tokens and dollars per stage, each model call with its task, and the finder's own spend separately.
5. Models come from OpenRouter, sorted by throughput, with routing preferences (p90 throughput of at least 40 tokens/s, p90 latency of at most 5 s) so a slow provider is tried last.

3. The life of a question

NeuralKG2 turns a question into a typed query, finds the tables that could answer each part, asks the right source for the numbers, combines them with code, and presents claims that each point at the receipt they came from. The same pipeline serves the Neural DB page (with or without a source restriction), the JSON /ask and the MCP tool.

3.1 The hosting layer (neuralkg2/space.py)

• A question is 1 to 400 characters (MAX_QUESTION, space.py:32).
• sources (for example datacommons) restricts the run to those ARD source families. The deployment wraps its catalog in SourceLimited (deployment.py:62, catalog.py:208), and _allowed (stages.py:364) is checked everywhere a plugin could be chosen, so no stage can reach a source the request excluded.
• Concurrency. A bounded semaphore (NEURALKG_CONCURRENCY, default 3) guards the pipeline; a request waits up to 30 s for a slot, then answers "busy". For a stream the worker thread owns the slot and releases it when the work ends; if the client disconnects the worker stops at the next stage boundary (ClientGone), so abandoned questions do not keep spending model money.
• Traces. Each run writes answer.json, annotated_query.json and a snapshot after every stage under NEURALKG_TRACES (default /tmp/runs), copied to NEURALKG_TRACE_BUCKET when set.

3.2 The run, end to end

flowchart TD
  Q[Question text] --> U[understand<br/>panel of 3 models]
  U -->|1-4 readings| L{next reading}
  L --> S[slots<br/>code]
  S --> D[discover<br/>ARD search per slot]
  D --> SEL[select<br/>plugin chooses documents<br/>may split meanings]
  SEL --> RE[resolve entities<br/>place, company, population]
  RE --> RA[resolve attributes<br/>bindings, breakdown plan]
  RA --> P[plan<br/>code, no network]
  P --> X[execute<br/>no model calls allowed]
  X --> C[combine<br/>code: compose, rank, correlate]
  C --> PR[present<br/>claims, evidence, charts, status]
  PR -->|complete| OUT[Answer]
  PR -->|partial or unanswered| L
  L -->|none left| BEST[best reading kept]
  BEST --> OUT

pipeline/run.py:answer (26-84) drives it. After understand, each reading is run through every stage in turn (STAGES, run.py:17-19). The first reading that ends complete is the answer. Otherwise the best of the readings (complete, then partial, then the rest) is kept, and the answer says which reading it used and why the others did not answer (run.py:72-77). This is the backtracking: it is by reading, not by entity. A source the user named that cannot be honoured ends it: no other reading may answer from elsewhere (run.py:67-68).

3.3 Stage by stage

For each stage: what it reads, what it does, who decides, what it records.

understand (understanding.py) - Reads: the question text. - Does: asks a panel of three models in parallel for a typed query. The task contract understand (v17) carries 13 query types (single value, comparison of entities, ranking, population value, breakdown, trend, difference, ratio, correlation, filtered list, item list, yes/no, vague aspect), their JSON shapes and worked examples. - Agreement (panel_readings, 317-413): the first two members are asked together. If their first readings have the same structure (type plus the shape of every field, ignoring wording, structure() 217), that settles it; otherwise the next member is asked, until two agree or the panel is used up. A tie goes to the simpler type. A slow member is not waited for: once one member has answered, the others get until PANEL_DEADLINE_SECONDS = 7.0 seconds after the start; the answer then goes with whoever answered and says so ("model had not answered 7.0 s after the start..."). Up to 4 readings are kept (MAX_READINGS), the first reading first, then readings other members gave instead, then computed alternatives; an alternative about a different subject is dropped (_same_subject). - Decides: models. Records: understanding_panel (votes and errors) and panel_notes in the trace.

slots (stages.py:128, SLOT_RULES 111-125) - Code only. One table maps each query type to the slots it needs. Slot kinds: value, population (the quantity for every member of a group), members (just the list), series, fact, categories (breakdown). A ratio gets two slots (numerator, denominator), a correlation two population slots, a filtered list one slot per condition.

discover (stages.py:203) - Does: one ARD search per slot (catalog.search), whose request carries the slot's own wording and the whole question as background (RemoteCatalog.request, catalog.py:141-158), plus the measure without its entity so a table found by the bare measure stays eligible. Plugins that know their term is thorny add their own searches (discovery_requests). Hits are grouped by source and annotated on the slot. If the finder's rerank discards every hit, the search is repeated without rerank (catalog.py:131-137). - A source the user names ("from NIH") is resolved to its publishers (publishers.py, task resolve_publishers) and limits the search to them; if it cannot be honoured, the slot is refused and nothing is searched elsewhere (stages.py:218-229). - Decides: ARD (see the ARD design document). Cost: one ARD search per slot, about 2 s.

select (stages.py:390-522) - Does: per slot and per source that can serve the slot's kind, the plugin chooses among its own documents with a bounded model call (choose_documents, plugins/base.py:327, task select_source_documents v12): up to 25 candidates, each described by the plugin's view; the model returns groups of documents, one group per distinct quantity (at most 6 groups of 5, base.py:324). Country-only sources choose knowing the place's kind. - Follow-up searches: documents the plugin prefers for the term are searched for before choosing (preferred_queries); its follow_up_queries may prompt a second choice over more documents; and the two parts of a ratio search for each other's dataset so both come from one dataset (_sibling_dataset_queries, stages.py:624). - If a term is ambiguous (GDP is nominal, real, per capita, PPP...) the plugin's refine_query splits the slot into one slot per meaning, each tagged with the published variable it stands for (slot:1.m1, slot:1.m2...). Every meaning is answered and labelled; none is preferred. The same attribute asked on the other side of a ratio is split the same way (shared), so the two sides can be paired like for like. - A meaning that names a known variable gets every source's document of it by an exact lookup (catalog.by_variable, ARD /variable), not a search. - Members slots skip this (the source lists members from its own structure). Breakdown slots pass all discovered documents to the resolve stage (their family, not a single document, matters). Fact slots are served by sources that publish facts about entities. - Only tables the search returned are used: a fact or members source must have an entry among the hits, and a slot whose returned tables cannot serve its kind fails with that reason (stages.py:407-426, 510-518). - Decides: models, bounded to the offered ids; code validates. Cost: one model call per slot per source (two when follow-up searches find more), about 1 to 3.5 s.

resolve_entities (stages.py:534-689) - Does: each source that chose documents resolves the slot's entity in its own way. For Data Commons: the place resolver, then the place's own types; when several places share the name, a bounded model choice (datacommons_choose_place, using the question for context, e.g. "California compared to New York" means the states). For a population slot ("counties of Texas"): the container and member type become a containment expression, with exact-name lookup of the container. For sources with no plugin, Wikidata resolves the entity; a plugin that cannot resolve it retries with Wikidata's identifier or name, and must still confirm it (stages.py:588-612). Open-data tables find the place among a place dimension's values by rule, with a bounded model choice (cube_place) for a name in the publisher's own words. - A named group no source knows ("New England") is listed by the panel (knowledge.list_members): two lists agree if they differ by at most one name in twenty; otherwise, when all three answered, a majority vote is used if every list is close to the majority list, with split and left-out members disclosed; a single model's list is used only when no other answered, and the answer says so (knowledge.py:116-145). The answer discloses that the list came from models. - Decides: source APIs; a model only for ambiguity. Cost: 0.1 s typical.

resolve_attributes (stages.py:932-968) - Does: turns each chosen document plus the resolved entity into a binding: an attribute (measure, unit, definition, calendar, grain, locator = variable and source facet), the identity, the plugin and its version. For breakdown slots it plans the split here, because models are forbidden later (section 3.6). Plugins may refuse a binding that cannot satisfy an explicit unit or direction. Open-data plugins choose a value for every remaining table dimension in one bounded call per table (cube_cells, plugins/cubes.py:223), checked against the table. - Decides: code, except breakdown planning and open-data cell choice (models).

plan (stages.py:1004) - Code only; no network, no model. Orders the requests to try per slot (the first is the plan, the rest fall back). For ratios and differences it prefers a compatible pair of bindings; for slots about the same attribute, one field every slot can use; for joined populations (correlation, filters), a key scheme every side shares (FIPS, ISO3). It records every decision. A relative year ("ten years ago") is recorded as dependent on an anchor slot.

execute (stages.py:1378) - Does: runs the planned requests: a value or series (compile_query, one HTTP request through the allow-listed transport, execute_query), a population's values, the members of a place, a breakdown's numbers. Each value result is verified against its request (verify_receipt) before it is accepted. - No model call and no semantic resolution may happen here: the phase guard raises (runtime.py:200-201, plugins/base.py:36-43). An explicitly requested year is passed to the source and a result for another year is refused (_run_population, requested_year). - Decides: code and the source API. Typical 0.05 to 1.1 s (measured 2026-10-09).

combine (stages.py:1778, per-type rules) - Code, with one bounded judgment. Ratio and difference pair answers by published variable, never by wording (_variable_key), then pass them through one check, _like_for_like: a model judges whether two figures about different entities are comparable (comparable_figures v3), and a current-price local-currency amount can be converted to dollars at the source's official exchange rate for its year (_to_usd). If no judgment can be had, nothing is composed. The composition itself (planning.compose) checks units, periods, populations and measure definitions. Rankings, correlations (Pearson and Spearman over the members both sides have), filters (numeric thresholds parsed once, numbers.threshold), yes/no and trends are plain code. A group of members summed to a named group is formed only if the members are one measure over one reporting span and the quantity adds up (declared by the plugin, else judged), and the total is recorded as a computation over its members' receipts.

present (stages.py:2218) - Does: per-type presenters write claims (each with the ids of the receipts or computations it rests on), disclosures (which meaning, which reading, conversions, model-listed members, years placed), failures, and the status. - Status rule: complete only when every slot is answered and the combine step produced the result its query type promises (COMBINATION_RESULT, stages.py:2058-2060) and nothing is missing from a series. partial when something answered but not everything; unanswered otherwise. - Evidence: receipts are collected from every slot, series, group member, breakdown part and conversion regardless of presenter; a claim that cites something the answer does not carry loses that citation and the answer says so (_evidence, 2195). - Charts: the plugin's own charts (for Data Commons, its published web components, plugins/dc_charts.py), generic Vega-Lite charts for populations, series, members, breakdowns and compared entities, validated and rendered with a CSP-safe interpreter (visualization.py:32-43). Each chart runs in a sandboxed frame without the page's origin (web/space.js:121-143). With defer_related, "related" charts from other sources are computed after the answer is delivered and sent as a later charts event, so they never delay the answer.

3.4 Annotated query: the state that flows through

AnnotatedQuery (pipeline/query.py) holds the typed query and, per slot id, a list of annotations per stage per source: ard_query, discover (resources), select (chosen documents), meanings, resolve_entity, resolve_attribute (bindings), plan (requests), execute (receipts, populations, members, categories), combine, and failure entries {stage, status, reason}. Because annotations are appended, never overwritten, every decision stays available to the presenter and the trace. A snapshot of the whole object is written after each stage.

3.5 Three worked examples (measured on the live service, 2026-10-09)

Seconds and dollars per stage, restricted to Data Commons, as of 2026-10-09 (before the open-data and named-source changes).

India's GDP. The panel reads it as an EntityValueQuery (entity India, attribute "gdp"). Discover finds Data Commons documents for many GDP variables. Select's document choice returns several groups, one per distinct quantity; the plugin splits the slot into one slot per meaning (nominal US$, constant, PPP, growth...). Each meaning is looked up for India and each is answered and labelled, with a disclosure "gdp is published as N distinct quantities; each is answered". One meaning lacking a value for the year makes the answer partial.

Ten most populous counties. A RankingQuery over a population (counties of the US). The plugin resolves the container to a containment expression, lists the ~3,200 counties' populations in a handful of observation requests, ranks them in code, and names only the ten shown (names are fetched lazily for big populations, named=False). Ranking, direction and count come from the reading's own fields.

Age distribution. A BreakdownQuery. Resolve plans the split with models (section 3.6), which is why that stage costs 6.9 s here; execute then needs one observation request and re-checks the arithmetic.

3.6 Breakdowns: why planning is in resolve, not execute

A breakdown needs two model choices, so they must happen where models are allowed. In resolve_attributes the Data Commons plugin (categories_plan):

1. takes the quantity's family (population type, measured property, statistic) from the discovered documents and asks ARD /variables which dimensions that family is split by;
2. asks a model to choose the option that matches what the user named ("age", "sex", "race");
3. lists the variables split by exactly those dimensions plus the unsplit totals, reads the newest observation of each for the place, and groups them by (source facet, date) so every number is one source's, for one date;
4. asks a model to choose a total and non-overlapping categories from the richest group, with up to two corrections when the arithmetic fails (the model is told "the categories add up to X but the total is Y: some overlap / some are missing");
5. accepts the split only if the categories add up to the total within 2% on every date the source has all of them (a sum that matches on one date can be a coincidence).

execute then re-reads exactly those variables for that source and date and re-checks the sum; the numbers become receipts the claims cite. A ranking or limit in the question (HighToLow, number) only chooses which categories to show, after the whole partition was validated.

3.7 Where models are used in NeuralKG2

Nothing else is a model: not the ARD request, the plan, the HTTP requests, the arithmetic, the status or the evidence.

3.8 Budgets and accounting

Budget (runtime.py:41): at most 20 model calls, $1.00, 90 s of cumulative model time per run, with reservations settled from reported token usage and the configured prices. A reply that crosses the time limit is kept and the crossing is traced. The answer carries a cost breakdown by stage and model, listing each model call with its task (pipeline/usage.py:35), and ARD's own spend per search is reported separately.

3.9 Failure behaviour

A failure is an annotation {stage, source, status, reason} shown to the reader. The common ones:

4. Latency, cost and quality

Measured by the status ledger on 2026-10-09 (neuralweb.dev/status), while other evaluations were running against the same service, so higher than the quiet-service figures in section 3.5. These figures predate the open-data plugins and named-source handling and have not been re-measured.

Where the time went (2026-10-09): understanding about 2 s, one ARD search per slot about 2 s, one document choice per slot per source 1 to 4 s, data fetching 0.05 to 1.1 s. Breakdown planning can reach about 7 s. The 7-second panel deadline, the provider throughput preferences and ARD's empty-completion retry were each added after one of these steps was observed taking 30 s or more.

5. Invariants that must hold

Each is enforced in code and covered by a test.

1. No model call and no meaning resolution in execute (runtime.py:200-201; tests/test_datacommons.py, test_models_are_not_asked_during_execution).
2. No composition without a like-for-like judgment between figures about different entities, and none from an unknown (_like_for_like, stages.py:1517; LikeForLikeTest).
3. A request restricted to named sources never reaches another source, in any stage and for every slot kind (_allowed, SourceLimited; SourcesTest). A source named in the question that cannot be honoured stops the other readings (RefusedSourceDoesNotBacktrackTest).
4. A requested year is the year returned, or the source is refused for that question (_run_population; ExplicitYearTest).
5. A claim cites only what the answer carries (_evidence; EvidenceTest).
6. complete means the combine step produced the result its query type promises (COMBINATION_RESULT; CompletionTest). A bookkeeping annotation is never a result.
7. A breakdown's categories add up to its total on every date the source has all of them (categories_plan; CoincidentalSumTest).
8. A stream's worker owns its concurrency slot (StreamSlotTests).
9. The status page tests every sample question and records when (tools/status_ledger.py).

6. What is open

• Continents and regions as a place type (needs a declared aggregation rule, such as a population-weighted mean).
• Source, year and definition shown on lists, rankings and filters.
• One meaning with no value makes a GDP question partial even when the others answered.
• Educational-attainment breakdowns and some lists (New England, Central America, Florida's cities) do not answer; vague questions ("How is Brazil's economy doing?") answer only in part.
• Population and ranking claims cite no receipts (a population result is not a receipt).
• Units on thresholds ("20%" against a fraction).
• The CLI, local server and frozen evaluator use Deployment.ask(), the hosted page uses ask_pipeline(); the two engines should become one.
• /health counters are per instance and reset on restart. The server also maps /healthz, but Cloud Run's front end may answer that path itself, so use /health.

7. File map