Research prototype · automotive validation
From crash evidence to traceable validation scenarios.
CalibraScene is an evidence-bounded AI workflow. It turns heterogeneous crash evidence into scenario-level representations and uncertainty-aware validation coverage decisions.
Missing evidence stays visible. Conflicts stay explicit. Only target-relevant uncertainty can hold a decision open; decisive claims keep their provenance.
Why this exists
A confident answer is not the same as a supported one.
Crash evidence arrives incomplete. That is normal — records are coded for other purposes, narratives skip details, and some facts were never measured. The risk is what happens next: a generative system asked to produce a complete scenario will usually produce one, filling the gaps with values that are plausible rather than supported.
In validation planning that matters more than it first appears. Coverage decisions are comparisons against parameter bands. An invented value is specific enough to land inside or outside a band, so an unsupported number does not just add noise — it can flip a decision from “unresolved” to a confident match or a confident exclusion. The engineer downstream has no way to see that the deciding value was never in the evidence.
Forced completion
- Counterpart motionstopped
- Path relationsame lane
- Impact speed42 km/h
Decided by a value with no evidence behind it.
Apparently complete. The decisive value was never observed.
Evidence-bounded
- Counterpart motionstoppedOBSERVED
- Path relationsame laneDERIVED
- Impact speednot recordedUNKNOWN
Held open until the speed evidence is recovered or adjudicated.
Visibly incomplete. The gap is the finding, and it is actionable.
Illustrative comparison. Synthetic values, not a crash reconstruction.
How it works
Three stages, one evidence spine.
Evidence enters as records from different sources and leaves as a decision about validation coverage. What matters is that nothing loses its epistemic identity along the way: a value that was inferred at stage two is still marked inferred when it decides something at stage three.
C1: Functional scenario
Heterogeneous crash evidence, normalised into recurring scenario structure.
Structured national crash records are parsed by explicit adapters rather than by a language model — the schemas are known, so generative extraction would add risk without adding information. Records are normalised into a common taxonomy so that scenarios from different sources become comparable.
- Source adapters
- Normalisation
- Recurring structure
C2: Logical scenario + SSR
Bounded enrichment into a structured scenario representation.
Unstructured evidence — narratives, diagrams, images — is where generative extraction earns its place. Candidates it proposes are admitted as assertions with a source role and an epistemic state, never as bare facts. Actors, road and environment, events and parameter domains are assembled with uncertainty and provenance attached.
- Actors
- Road & environment
- Events
- Parameter domains
- Uncertainty
- Provenance
C3: Validation coverage
Dimension-by-dimension reasoning against versioned catalogue constraints.
Each validation target evaluates only the dimensions it constrains. Missing or conflicting evidence on a required dimension yields UNKNOWN or CONFLICT rather than a guess; unresolved evidence outside that target does not contaminate the decision, and an independently supported contradiction can still justify EXCLUDED.
- MATCH
- EXCLUDED
- UNKNOWN
- CONFLICT
Evidence and provenance spineSource, epistemic state and provenance travel with every assertion through all three stages.
Interactive demo
Change the evidence. Watch the decision move.
A working model of the decision semantics, running entirely in your browser. Pick a scenario, then add or withdraw evidence and see how the coverage decision responds. Nothing you do here leaves the page.
Ego vehicle approaches a stationary vehicle in the same lane. Impact speed decides whether the case falls inside the target’s tested band.
Change the evidence
Coverage decisions
- Target AMATCH
Same-lane stationary counterpart, low-speed band.
Supported match. Every decisive dimension is positively supported by admissible evidence.
- Decisive
- Counterpart road user
- Counterpart motion
- Path relation
- Impact speed
- Target BEXCLUDED
Same-lane stationary counterpart, higher-speed band.
Supported exclusion. A supported value contradicts this target. The exclusion stands on evidence.
- Decisive
- Impact speed
Why does the system know this?
Select a parameter to trace its provenance.
Resolved parameters
Evidence ledger
- Counterpart road userpassenger carHistorical observationDecision eligibleSource A — structured collision record
- Ego motiontravelling forwardHistorical observationDecision eligibleSource A — structured collision record
- Counterpart motionstoppedHistorical observationDecision eligibleSource A — structured collision record
- Path relationsame lane, same directionDeterministic derivationDecision eligibleDerived from ego and counterpart headings
- Road contextsingle carriagewayHistorical observationDecision eligibleSource B — narrative report
- Visibilitydaylight, clearHistorical observationDecision eligibleSource B — narrative report
- Impact speed28–34 km/hHistorical observationDecision eligibleSource A — recorded impact speed band
Synthetic demonstration. It illustrates CalibraScene’s evidence and decision semantics; it is not a crash reconstruction, safety certification or empirical accuracy benchmark.
Target A: MATCH. Target B: EXCLUDEDCurrent evidence
What the system demonstrably does.
The prototype is implemented and under active test. Rather than publish headline figures that will be revised at the next research freeze, this section describes the behaviour that holds across releases — and states plainly where the evidence stops.
- Regression-testedPrototype implementation
A working prototype, held in place by tests
The private research implementation is covered by automated regression, schema and evidence-contract checks, so a change that breaks the evidence semantics fails in the test suite rather than in a demo. Exact release figures are versioned with each frozen evidence snapshot and will be published here after the next freeze.
- DeterministicReproducibility
A deterministic evidence path
The deterministic stages are built to return the same structured outputs and the same validation decisions from the same pinned inputs and configuration. Model, prompt, retrieval set and configuration are versioned, so a result can be traced back to the conditions that produced it.
- Evidence-boundedEvidence handling
Target-relative fail-closed semantics
Missing, conflicting or ineligible evidence cannot settle a target dimension that depends on it. Uncertainty elsewhere stays visible but does not automatically block unrelated targets; each result retains the constrained dimensions that were decisive or unresolved.
- ImplementedEnd-to-end integration
The C1 → C2 → C3 path runs end to end
The private prototype connects functional-scenario processing, scenario-level evidence representation and deterministic validation-coverage reasoning as one path rather than three disconnected experiments. This page describes that architecture without publishing private datasets or implementation logic.
Failure-mode work
Does unsupported completion create false exclusions?
A deliberately small frozen experiment asks whether a policy that fills a missing value with a plausible default buys its extra completeness by producing confident, wrong exclusions — the specific failure this project exists to prevent. It is designed as a feasibility and failure-mode probe, not as a performance estimate.
Small-sample failure-mode work of this kind cannot support a population-level accuracy claim, and is not presented as one. Figures will be published once an approved evidence release carries them.
Detailed benchmark figures are versioned with frozen research releases. This page reports demonstrated system behaviour and evidence boundaries rather than provisional headline metrics. Larger semantic-accuracy and domain-validation claims remain intentionally unreported until independently adjudicated evaluation is available.
Evidence architecture
What a scenario is, structurally.
The structured scenario representation is the artefact everything else is defined against. Its shape carries the guarantees: epistemic state is stored separately from source role, and parameters that depend on each other stay joined.
Select a node for detail.
- Scenario
Sources
Each source is registered with its version and access conditions. An assertion always points back to one.
State and role are separate
How confident we are and where a value came from are two different questions. Storing them separately is what lets a protocol default be recorded without becoming an observation.
Unknown is target-relative
A parameter with no admissible support blocks only a target whose constraints depend on that parameter. Unrelated targets are evaluated from their own constrained dimensions; a separately supported contradiction can still justify EXCLUDED.
Conflict stays conflict
Disagreeing sources are not averaged, ranked by confidence, or silently resolved. The disagreement is the output.
Joint stays joint
Dependent parameters remain tuples. Expanding them into a Cartesian product would invent combinations that no case supports.
Where the generative AI is
Generative where it helps, deterministic where it must be.
The architecture is hybrid on purpose. Structured national crash databases have known schemas, so they are parsed by explicit adapters — handing them to a language model would add failure modes without adding capability. Generative and multimodal models are used where evidence is unstructured and semantic reading is genuinely hard.
The division of labour is the point. Generative models propose; the evidence layer decides what a proposal is allowed to become. A model reading a narrative can suggest a speed band, and that suggestion is admitted as a bounded, model-inferred assertion. It cannot become an observation, and it cannot narrow itself into a value the source never supported.
Generative extraction
VLM and retrieval-grounded reading of narratives, diagrams and images.
Evidence admission
Source role, provenance and entailment checks. Unsupported narrowing is rejected.
Scenario representation
Assertions assembled into the SSR with state and uncertainty intact.
Deterministic reasoning
Catalogue comparison and coverage decisions, reproducible run to run.
The evidence and decision layers described here are implemented in the current prototype. The broader generative and multimodal extraction layer is target architecture for the Advance phase: it is described as intended, not as measured, and no claim is made that a production VLM or retrieval layer has been evaluated at this stage.
Design controls
Built to be checkable.
These are engineering controls rather than principles. Each one is a property of the representation, which is why they hold under pressure instead of relying on a model behaving well.
Useful uncertainty
The system returns UNKNOWN when a target-relevant dimension lacks admissible support. Missing evidence outside that target remains visible but does not veto an otherwise supported decision.
Visible disagreement
Conflicting evidence stays CONFLICTING. Nothing is averaged into a false consensus or quietly decided by whichever source was read last.
Provenance by construction
A decisive parameter retains a reference to the evidence or the deterministic rule behind it. The trace is not reconstructed after the fact.
Explicit review state
Human adjudication is represented in the record, so reviewed and unreviewed assertions are never confused.
Separated source roles
Model inference, historical observation, external context, deterministic derivation and protocol values stay distinguishable for the life of the record.
CalibraScene adopts high-assurance design controls appropriate to safety-oriented engineering use. Final regulatory classification depends on intended deployment.
Current status
TRL3 · v0.21TRL3 research prototype.
CalibraScene is a working research prototype implementing the C1 → C2 → C3 evidence path: functional-scenario processing, scenario-level evidence representation and deterministic validation-coverage reasoning. The private research package is under active technical and domain review.
Current focus
- External technical critique of evidence sufficiency and catalogue semantics.
- Stronger, independently adjudicated evaluation.
- Consolidating the next frozen evidence pack.
- Advance-phase integration planning.
Not claimed
CalibraScene does not claim homologation, Euro NCAP certification, crash-reconstruction authority, deployment safety or a complete automotive safety case. It is decision support for validation preparation, and it is early.
Context
AI-BOOST Challenge 4
CalibraScene is being prepared as an application to AI-BOOST Challenge 4: Generative AI for Automatic Test Case Generation from Crash Databases & Standards. Selection is pending.
CalibraScene is an independent project and is not presented as funded, endorsed or certified by AI-BOOST, Siemens or Euro NCAP.
About
An independent research and engineering project.
CalibraScene is built by Tomi Šeregi. The implementation and its evidence archive are private; this page describes the architecture and semantics rather than publishing the system.