Mechanistic interpretability for the EU AI Act

Prove what your model actually did.

Glassbox traces the circuit behind a decision, measures how faithful that explanation is, and drafts Annex IV documentation from it. One function call, on open-weight models.

Live demo
Open source · MIT core · built for GPT-2 → Llama-3, Mistral, Phi-3
Grounded in
Built on PyTorch & TransformerLens Method anchored to ACDC (Conmy 2023) & IOI (Wang 2022) 1,716 tests passing arXiv 2603.09988
The method

Proof from the model's own wiring,
not a story told afterward

Most explainability tools describe a model from the outside. Glassbox reads the inside. It ranks every attention head by the effect it has on the output, then keeps only the heads the decision depends on.

01

Discover

Attribution patching scores each attention head by its causal effect on the prediction. Three forward passes, no correlation guesswork.

15–37× faster than ACDC · 1.8s on a laptop
02

Verify

Sufficiency and comprehensiveness isolate the smallest circuit that still reproduces the behaviour, scored against full ablation. The explanation is measured, not asserted.

credit task (gpt2) · F1 0.00 · Grade C · CHECKS FAILED — no faithful circuit
03

Document

The nine Annex IV sections are laid out from the analysis. Fields the measurement supports are filled; fields only the provider can supply, including the §8 declaration of conformity, are marked for you to complete.

9 sections laid out · measured fields filled · the rest marked
Measured results

Numbers you can reproduce

15–37×
faster than ACDC
across the GPT-2 family
0.704
faithfulness F1, Grade B
IOI task (gpt2), vs full ablation
9
Annex IV sections laid out
provider-only fields marked
0.009
confidence ↔ faithfulness
correlation (r), orthogonal

Wall-clock timings from gb.analyze() on Apple M1 Pro, weights pre-loaded; ACDC baseline per Conmy et al. 2023. Full methodology, hardware specs and raw data in BENCHMARKS.md — reproduce with scripts/benchmark.py. Every approximation is disclosed in the result dict.

From prompt to evidence

One call. Many kinds of decision.

The same call accepts any prompt and any set of answer tokens, and picks a counterfactual when you let it. Whether a faithful circuit exists is what it measures: raw GPT-2 has none for credit decisions, and Glassbox says so.

audit.py
# faithfulness is measured, not assumed
from glassbox import GlassboxV2, build_annex_iv_vault

gb = GlassboxV2(model)
result = gb.analyze(
    prompt="Credit score 620. Decision:",
    correct=[" approved", " approve", " yes"],
    incorrect=[" denied", " deny", " no"],
    certify="hessian",
)
result["faithfulness"]["f1"]   # 0.00 — gpt2 has no faithful credit circuit
result["evidence_tier"]["tier"]  # self-graded
vault = build_annex_iv_vault(result)
1

Point it at a model

Load a supported open-weight transformer. No fine-tuning and no labels.

2

Run one call

It finds the circuit, verifies the counterfactual actually moved the decision, and scores how faithful the explanation is — then grades its own evidence tier.

3

Get the vault

A draft Annex IV record for your own review, completion and sign-off.

EU AI Act · Annex IV

Annex IV documentation,
drafted from measurement

Providers of high-risk AI systems must keep Annex IV technical documentation (Article 11 of Regulation (EU) 2024/1689). Application dates and penalties were amended by Regulation (EU) 2026/1744; check the current legal text. Glassbox drafts the parts of that documentation a measurement can support.

§8
the declaration of conformity is never auto-filled: it is the provider's signed statement.
9 sections
laid out in the Annex IV structure; fields only you can supply are marked [PROVIDER TO COMPLETE].

What you actually hand over

Every run produces a structured Annex IV draft as JSON. Measured evidence (the circuit, faithfulness metrics and their evidence tier) is filled in; data governance, risk measures and the declaration are left for you, and marked so nothing is silently skipped.

  • Measured fields filled; provider-only fields marked to complete
  • Faithfulness metrics included with their evidence-tier disclosure
  • Exported as JSON for your technical file
Art. 9 riskArt. 13 transparencyArt. 15 accuracyArt. 72 post-market
annex_iv_vault.jsonexample output
IOI example · model gpt2 · AnnexIVReport
§1General description of the system filled
§2Development and design, incl. circuit evidence filled
§3Monitoring, functioning and controlpartly · your input
§4Data and data governanceyour input
§5Risk managementpartly · your input
§8EU declaration of conformityyour signature
IOI exampleF1 0.704grade Bprovider fields marked
Licensing

The core is open.
Commercial use is licensed.

Circuit discovery and faithfulness metrics are MIT-licensed. The compliance engine and the investigation layer are under the Business Source License 1.1: free inside your own organisation, licensed when offered to others.

Open source

MIT core

Free

For researchers and engineers exploring circuits.

  • Circuit discovery and faithfulness metrics
  • CLI and Python API
  • Community support on GitHub
pip install glassbox-mech-interp
Your own AI systems

Internal use

Free

BSL 1.1 components, used inside your own organisation.

  • Annex IV documentation drafts for your own systems
  • Investigation of your own AI systems
  • Each version becomes Apache-2.0 after four years
Read the license map
Offering it to others

Commercial license

On request

For products or services built on the BSL components.

  • Compliance, governance or investigation services
  • Redistribution and hosted use
  • Design-partner pilots by agreement
Contact

This summary is not the license text; LICENSE, LICENSE-COMMERCIAL and NOTICE in the repository govern. No hosted service is offered at present.

How Glassbox compares

Faithfulness is the line
most tools can't cross

Black-box explainers describe a model from the outside and can't tell you whether the explanation is true. Circuit tools can, but they're slow. Glassbox keeps the rigour and drops the cost.

ApproachFaithfulness measured?Needs open weights?Speed (GPT-2)
Glassbox attribution patchingYes (suff / comp / F1)Yes1.8s
ACDC (Conmy 2023)YesYes~65s · ~37× slower
SHAP / LIME (black-box)No guaranteeNo (works on APIs)varies
Confidence scoresNo (r = 0.009)Noinstant

Each row is a different trade-off, not a verdict. Black-box methods are the right tool when you only have API access — Glassbox ships a black-box auditor for exactly that case. Where the weights are available, it gives you ACDC-grade circuits at a fraction of the runtime, and unlike a confidence score, an explanation whose faithfulness is actually measured.

Model coverage

Validated GPT-2 → 12B.
Architected for more.

Nine architecture families validated end-to-end (82M → 12B), with grouped-query attention and RMSNorm handled correctly — GPT-2, Pythia/GPT-NeoX, GPT-Neo, OPT, Llama-3, Mistral, Gemma-2, Qwen2/2.5, Yi (Llama-architecture) and Phi-3 — ten model series across nine distinct architectures. Beyond ~13B, gradient-based attribution needs multi-GPU memory; that path is implemented but not yet validated live.

ClassParameter rangeStrategyStatus
small / medium82M to 12Bstandardvalidated — 9 families
large13B to 70Bcheckpointneeds multi-GPU · pending validation
xlarge / xxlarge70B to 200B+checkpoint + offloadimplemented · unverified (cluster)

Reproducible in VALIDATION_LOG.md; comprehensiveness is specificity-checked against a random same-size circuit at every scale. Circuit-level analysis requires open weights — closed API models (Claude, GPT, Gemini) can only receive black-box documentation support, because faithfulness can't be measured without access to the weights.

Beyond name-swap tasks

Decision prompts, not just puzzles

Glassbox picks a counterfactual that fits the prompt in front of it, so you can point it at decision-style prompts. Whether the model has a faithful circuit for that decision is then measured, not assumed.

Credit

Loan affordability and limit decisions

example

Medical triage

Priority and severity classification

example

Employment

CV screening and candidate ranking

example

Corruption strategies

name-swap, antonym, semantic negation, random-token fallback

auto

A confident answer is not the same as a correct one.

Across our runs, model confidence and circuit faithfulness barely correlate, at r = 0.009. That's why a compliance auditor needs the circuit, not the score.
Research & releases

The work behind the tool

2026 Causally grounded mechanistic interpretability with faithful explanationsAttribution patching, faithfulness metrics, and the gap between confidence and faithfulness. Mahale, 2026. Paper v4.8.0 One grade and one status everywhere + agent failure typesThe report, audit log, MLflow, model card and MCP server now share one grading rule; MLflow and model cards no longer say “checks passed” on sufficiency alone; failing agent runs get a failure type; outputs show the real version. Release v4.7.0 Annex IV point 4 fixed + direction-aware investigationPoint 4 is now the appropriateness of the performance metrics, with data under point 2(d); status values say “Glassbox checks”, not compliance; no legal deadlines in generated drafts; interventions now report which changes hurt and which helped. Release v4.6.1 Accurate Annex IV drafts + supply chainModel card fixed and stripped of overclaims; evidence vault numbered as Annex IV points 1–9; provider-only fields never asserted; trusted publishing with attestations; licence audit in CI. Release v4.6.0 Investigation layer on PyPI + packaging fixFirst release containing glassbox.v7 (BSL 1.1). The wheel now installs only the glassbox package; 4.5.1 also installed stray top-level packages, including one named mcp. Release v4.5.1 Scale-aware exact circuits + reconciled benchmarksPublished exact-sufficiency circuit selection (analyze(exact_circuit=True)); one measured canonical number set across code, docs and site; corrected the large-model parameter estimate. Release v4.5.0 Auditable interface + conformance gateOne conformance suite makes any architecture auditable; post-market drift monitoring (Art. 72) and NIST AI RMF / ISO 42001 report packs from the same audit. Validated on GPT-2 Small. Release v4.4.0 Verbalizer sets + evidence tiersMulti-token answer sets via logsumexp pooling, a counterfactual check on every run, self-graded evidence tiers A–D. Release Note Why confidence isn't a compliance proxyThe orthogonality result (r = 0.009) and what it means for auditors relying on model confidence. Engineering

Stay in the loop as Glassbox develops.

The open-source core is on PyPI today. Leave your email to hear about releases and design-partner pilots. No hosted service, price or launch date is promised.

Prefer to talk first? Email the author. Glassbox does not give legal advice or classify systems under the AI Act.

Get release updates

Privacy notice

The waitlist form collects your email address (and anything you choose to add) so we can send you Glassbox release updates and, if you ask, information about pilots — that's the only purpose. Data controller: Ajay Pravin Mahale (contact: mahale.ajay01@gmail.com). Processing happens on Vercel (hosting), Resend (email delivery) and, if enabled, Upstash (storage), acting as processors. We don't sell or share your address, don't use it for third-party marketing, and delete it on request or once it's no longer needed. You can ask for access, correction, or deletion at any time by emailing the address above (GDPR Arts. 15–17). This site sets no cookies and loads no third-party fonts or trackers. Full privacy policy.