System Architecture · stable local authority 1.34.0
Read the workbook.
Separate evidence from meaning.
Keep people in control.
Excel Intelligence & Controls now has two explicit runtime layers. The public Browser Analyzer maps structure, relative formula families, dependency evidence, Excel errors, and review signals entirely on the device. The separately installed 1.34.0 Agent uses five Skills, Python, and SQLite to build candidate semantics, standardize governed data, answer traceable queries, run reversible What-If, and prepare handover or RDF/OWL outputs. Machine checks do not approve business meaning. Human review remains required, and this static website does not host the full Agent, database connections, or model credentials.
Current product workflow
Five Skills share one evidence and governance boundary.
The Skills are separate responsibilities, not five competing sources of truth. The project SQLite workspace remains the machine authority; candidate, machine-verified, human-confirmed, and published states are not interchangeable.
| Skill | What it does | What it does not claim |
|---|---|---|
Build | Scopes sources, extracts workbook evidence, forms candidate semantics, records expert decisions, and validates population closure. | Does not approve business meaning by itself. |
Standardize | Applies eligible definitions to produce standard data, reconciliation outputs, and controlled workbooks. | Does not rewrite ontology definitions or source facts. |
Query | Returns definitions, values, trust, trace, attribution, reversible What-If, handover, and RDF/OWL exports. | Does not bypass governance or mutate the workspace. |
Enterprise Management | Manages enterprise candidates, revisions, exchange, and project-adoption handoffs. | Does not read Excel or perform GitHub release actions. |
Ontology Git | Handles reviewed branch, PR, merge, Release, discovery, and download workflows. | Does not interpret semantics or write project SQLite. |
Preserved method foundation
The earlier six-step method still explains where the product began.
The map below is retained as historical method evidence for the browser and original workbook-reconstruction workflow. It is not the current 1.34 CLI or Skill inventory; the current five-Skill responsibility map is above.
System architecture map (scroll horizontally on narrow screens)
First Principles
Two principles. Two gates.
Two gates limit when the system may draw a conclusion. The machine enforces one gate; a person enforces the other.
Gate One · MACHINE GATE
Does it calculate correctly?
To advance to approved, the machine criteria must hold: status ∈ {verified, unverifiable}
and the evidence chain must not be empty.
Failing this gate does not reject ingestion. It places a ceiling on the element: the element still enters the registry, but can advance only as far as
proposed until verification succeeds. Gate One is a height limit, not a barrier.
Gate Two · HUMAN GATE
Does it mean what the business says it means?
To advance to published, the human criteria must hold: at least one review record with
verdict = approved, a non-empty owner, and a publisher who is not a digital worker.
AI cannot publish its own output. The code checks the publisher's identity.
Two States · TWO AXES
Neither overwrites the other
status records the machine view (verified / failed / unverified / unverifiable), while
reviewState records the human view (draft → proposed → in_review → approved → published → deprecated).
The two axes cannot rewrite each other: an expert's approval does not turn a failed regression into a pass, and a passed regression does not equal business approval.
False Green · FALSE GREEN
A rule can pass without testing any rows
Misspell a column name in a rule and the rule may test zero rows while the report still returns a clean ✅ and exit code 0. “Not tested” and “tested successfully” then look identical in the report.
When an alias cannot be resolved, verify refuses to run and names the specific rule. It returns exit code 2, not 0 or 1.
No Guessing · NO GUESSING
Fuzzy matching is harmful here
If a rule's “Amount” is guessed to mean the workbook's “Amount incl. tax,” the rule can test the wrong column without an error. Column alignment therefore uses exact matching only (NFKC normalization plus character-by-character equality after punctuation removal). Anything unresolved is reported for human review.
Reuse · CONFIRMED KNOWLEDGE
Extract → review → preserve → reuse
A definition confirmed this period is retrieved automatically by context next period and written directly into the rules by scaffold,
without the analyst rewriting it.
Measure reuse by whether the share of questions that still require a human answer falls with each cycle.
Analysis Lifecycle
Eight steps, each with an inspectable file.
Eight steps turn a workbook that nobody can fully explain into conclusions ready for review. Every step saves a file that another reviewer can inspect.
Profile: What kind of workbook is this?
Keyword scoring compares the workbook against nine archetypes, then infers the header depth, primary-key columns, dimensions, measures, and built-in validation columns. Header depth is the easiest inference to get wrong and the foundation for everything that follows.
Structure: Turn the workbook into readable text
Multi-row headers, merged cells, and formula columns become structured Markdown while formulas remain intact, because formulas are the source code of this legacy system. Cross-workbook references are scanned at the same time to flag broken links and circular references.
Mask: Separate structure from values
An external model may receive the workbook's structure, but not its values. In the structure package produced by mask,
column names are replaced and values are synthetic. The de-anonymization map is stored separately in local_only/ and never travels with the package.
The MANIFEST includes a self-check: raw-value occurrences must equal 0, or the check fails.
Retrieve: Bring back confirmed knowledge
Before writing rules for the current period, retrieve the domain's established definitions from the ontology registry.
Only published knowledge counts. Expert agreement alone is not enough: the knowledge must pass both gates
and be formally published by an ontology administrator before the next analysis can reuse it automatically. This is the Second Principle in operation.
Model: Generate the rule scaffold and align aliases
The scaffold turns profile results and retrieved knowledge into a spec skeleton. The critical action is alias alignment: the saved mapping between each alias and a column must be resolved against the current workbook's actual column names. Any mismatch is named explicitly. Fuzzy matching could create the next false green.
Trace: Formula chains and cell-level lineage
Formulas become an attribute DAG that shows which inputs feed each result column and how many layers they cross.
blame asks the inverse question: why does this cell have this value? It traces dependencies back to the source
and exposes cells that were manually overwritten outside the formula chain.
Verify: Recalculate every rule on the full dataset
This is the center of the system. Nineteen workbook-level check types cover uniqueness, non-null values, ranges, balances, roll-forwards, cross-sheet reconciliation, temporal foreign keys, version differences, and more; row-level checks recalculate derived formulas row by row. Anything that does not pass this gate is excluded from every conclusion.
Deliver and preserve: Conclusions, questions, and registry entries
Verified elements produce an annotated result workbook, breakpoint dashboard, and analysis report. Anything unverified or undecidable by the machine becomes a question list for a business expert. Only expert-confirmed knowledge enters the registry; only knowledge that passes both gates can be published and retrieved automatically in Step 04 next period.
|| true.
0 = completed and clean; 1 = completed with issues found (“human attention required,” not “crashed”);
2 = refused to run because a precondition failed, such as an unresolved alias. Refusal is not failure; it prevents an untrustworthy green light.
4 = masking self-check failed. Treating 1 and 2 as the same failure, or swallowing both, breaks this contract.Ontology → System
Five ontology layers. Five system layers.
A mature business workbook is already an implicit system: its data model, validation rules, calculation logic, and operating process all exist, but they are embedded in cells and verbal conventions. Reverse engineering makes the implicit explicit. Once enough is explicit, the system can be derived mechanically from the ontology.
| Ontology layer | Derived system component | Current local prototype |
|---|---|---|
| ① Objects + ② Attributes (input layer) | Data layer: table DDL / single-file database / input-form schema | profile · scaffold |
| ② Attribute DAG (derived) | Calculation layer: recalculation engine evaluated in topological order | graph · analyze --what_if |
| ③ Relationships | Data layer: foreign-key constraints and joins; interaction-layer suggestions and dropdowns | links · reconcile |
| ④ Logic | Validation layer: pre-save validation API plus scheduled full-dataset checks | verify |
| ⑤ Actions | Workflow layer: check preconditions → execute → audit log → trigger recalculation | Not built (what_if is its read-only precursor) |
| ①–④ combined | Interaction layer: object detail / breakpoints and exceptions / impact view | dashboard |
Historical command map
The original method commands. Preserved for provenance.
This table documents the earlier single-tool implementation behind the saved synthetic pipeline demo; it is not the current 1.34 public interface. In that historical baseline, excel_ai.py is 7,522 lines and has only two third-party dependencies,
pandas and openpyxl; everything else uses the standard library. There are no network calls and no telemetry.
Commands hand off through files, and every intermediate artifact can be opened independently for inspection. That is why the full pipeline can run offline.
| Subcommand | Stage | Responsibility |
|---|---|---|
| Profiling and structuring | ||
profile | Step 0 | Archetype detection · header depth · keys / dimensions / measures / validation columns · structural shape |
tomd | Step 0 | Convert to structured Markdown while preserving formulas |
links | Step 0 | Cross-workbook reference graph · broken links · circular references (returns 1 when issues are found) |
mask | Data boundary | Masked structure package · synthetic samples · MANIFEST self-check · mapping stored only in local_only/ |
| Understanding and modeling | ||
context | Flywheel · reuse | Load published ontology context—published only |
scaffold | Modeling | Generate spec scaffold · align aliases to current columns · refuse to run if unresolved |
graph | Step 4b | Lift formulas into an object graph and attribute DAG |
blame | E1 | Reverse fault isolation: after verification fails, trace the dependency chain to the deepest consistent node |
merge | E1 | Merge ontologies across workbooks: explicit sameAs first; heuristics produce candidates only |
| Verification | ||
verify | First Principle | Full-dataset regression: row-level checks plus 19 workbook-level check types |
reconcile | M | Reconcile two definitions: every difference includes a seven-level routing layer plus attribution candidates (candidates are not conclusions) |
sensitivity | E1 | Sensitivity ranking: perturb each input by 1% and rank by leverage |
restate | M | Organizational restatement: recalculate full transfers automatically; refuse automatic split restatement without an approved rule |
batch | M | Hundred-BU batch: one failure does not stop the batch; failures are marked in the summary |
| Delivery | ||
analyze | Step 5b | Group summaries / distributions / trends / outliers / rule exceptions / What-If (read-only) |
dashboard | Delivery | Single-file web dashboard for breakpoints and impact surface |
output | Step 5 | Color-annotated Excel output with lineage / ontology / verification summary |
questions | Flywheel · extract | Open-question list: spec TODOs + ontology openQuestions + unattributed workbook-level verify exceptions |
The ontology registry is a second standalone executable with 1,560 lines and only the standard library and no third-party package. It manages reviewed definitions and rules through seven subcommands covering the complete path from ingestion to publication:
| Registry subcommand | Flywheel stage | Responsibility |
|---|---|---|
import | Preserve | Ingest candidate elements while scanning for instance data; reject the entire batch if any is detected |
list | — | Query existing knowledge by domain, machine status, and review state |
review | Review | Record a review verdict: who, when, and whether it passed |
review-sheet | Review | Export a review sheet for business experts |
apply-reviews | Review | Apply completed review feedback to elements in a batch |
publish | Publish | Release only after Gate Two; a digital worker attempting publication is rejected immediately |
resolve | Reuse | Resolve an alias or term to a specific element for the next analysis |
ObjectType · AttributeType · Function ·
LinkType · LogicRule · ActionType · Term ·
Exception · DiffCase · Runbook · OpenQuestion.
The first seven map to the five ontology layers. The last four preserve the exception encountered this time, the attribution for this difference, the handling procedure for this issue,
and the unanswered question. Past pitfalls become reusable assets too.Data Boundary
Data stays inside.
Management reporting, compensation, and cost workbooks can contain a company's most sensitive information. The design keeps raw values out of any package that may leave the workspace.
01 · DUAL TRACK
Dual-track data discipline
Structure may leave; values may not. An external model can receive a masked structure package, but instance values such as specific amounts, names, and identification numbers always remain in the workspace. This is an architectural constraint, not a usage recommendation.
02 · SELF CHECK
The masked package checks itself
The structure package MANIFEST records raw_value_occurrences: the number of times a raw value appears in the package.
This number must be 0. Any other value fails the self-check with exit code 4. No shareable structure file is written;
only a refusal notice without raw values remains.
The self-check runs twice: first against strings in memory before writing, then again against bytes read back from disk, including the MANIFEST itself. The second pass is decisive because the files on disk, rather than the intermediate in-memory state, are what could leave the workspace.
03 · KEY SEPARATION
Store the de-anonymization map separately
The map that restores masked column names to real names is stored in local_only/.
It is the key to the masking scheme and never travels with the structure package. Sending both would defeat the masking.
04 · INGEST GUARD
Block instance data at ingestion
When knowledge enters the ontology registry, labels, definitions, quotations, and review notes are scanned. Suspected amounts or long numeric identifiers cause the whole batch to be rejected, with the field named. Reports include only digit count and magnitude; they never echo the raw value, which would copy the sensitive value into another log.
05 · BROWSER LOCAL
Analyze on drop, with zero upload
The homepage's profile analysis runs entirely inside your browser; the file never passes through a server. The page makes no cross-origin requests, and the Excel parser is hosted on the same origin. It also works offline.
06 · AIR-GAPPED
The command-line version can run fully offline
The complete pipeline needs no network connection: there are no remote calls, no telemetry, and all intermediate artifacts stay in local directories. A sensitive workbook can be processed end to end on an air-gapped machine.
Next Stop
Inspect the saved test or run the Browser Analyzer.
Choose a workbook in the Full Analyzer for a local structural profile, or open the saved synthetic run to inspect the complete workflow evidence.