Causal attribution for scenario testing
Your tests say a scenario fails.
We measure why.
Scenarios now come from fleet logs, crash reports and LLM generators: nobody knows them intimately any more. The Causal Map restores that knowledge — which actors a failure needs, which variables carry it, whether it was avoidable — measured in simulation.
Why now
The more we delegate design to AI, the less we understand what we built.
More and more scenarios arrive undesigned: logged, reported, generated.
Pass/fail says it failed; the Causal Map measures why, by re-simulation: whether the SUT could have avoided it, and what least change avoids it.
Simple: change one thing, replay the scenario, measure the change on your KPIs. Hard: knowing which changes to try. The Causal Map computes which ones to run.
Expertise erodes when understanding is delegated. Teams that stop seeing why their scenarios fail stop being the experts.
ISO 21448 (SOTIF) asks you to identify triggering conditions behind hazardous behaviour, including in scenarios nobody on your team designed.
Method
How it works
Scenario sources
- Fleet logs
- Scenario generators (including LLM-powered)
- Crash reports
- Suppliers, other teams, OEMs, regulators
Failure KPIs
Impact, TTC, RSS, Jerk…and user-defined KPIs
System under test
SUT / controllerThe system being evaluated
FAILURE ANALYSIS
QUESTIONS ANSWERED
- 1.Was it avoidable?
- 2.Which actors are necessary, alone or together?
- 3.What least change avoids it?
- 4.Did a broken traffic rule cause it?
- 5.When was it lost?
- 6.Which variables carry it?
- 7.How far can the scenario shrink and still fail?
Nothing fails yet? It first looks for a failure in the declared space, then explains what it finds.
Cases
Three sources, one analysis
One illustrative case per type of input: fleet logs, crash reports, LLM-generated. The same automated analysis has run on hundreds of scenarios.
A logged near-miss, from the Waymo Open Motion Dataset
Waymo 40213e4d, arterial, same direction, 8 actors: the partner moves into the lane of a recorded car; we hand that car to the SUT (an R157 reference driver).
Was it avoidable? yes: the SUT’s controller brakes in time, then accelerates again into it
Which actors are necessary, alone or together? the partner and A5; not A2, A3 and A6; A1 together with A5 under three KPIs, the Waymo car under one KPI only
What least change avoids it? the partner’s initial speed, −0.67 or +1.34 m/s: it fails only between 11.4 and 14.1 m/s
Did a broken traffic rule cause it? none: the files declare no traffic rule to break
When was it lost? 0.79 s before the contact
Which variables carry it? the partner’s {speed}, {braking}, {onset}, {speed, onset}
How far can the scenario shrink and still fail? without A2, A3 and A6: same outcome, 5 of 8 road users kept; the partner’s speed fails only between 11.4 and 14.1 m/s: search space −62 %
Case: Waymo Open Motion Dataset, scenario 40213e4d3bdc6f75, https://waymo.com/open/
A real crash, from an NHTSA crash report
Case NMVCCS 2007-03-018, 4 actors: the SUV (the SUT) hits a left-turning motorcycle (V2), then a pole and a pedestrian.
“The crash occurred in a signalized intersection of two roadways. […] V1, a 2002 Ford Expedition sport utility vehicle with one occupant, was traveling southbound in lane two. V2, a Harley-Davidson motorcycle with two occupants, was traveling westbound in lane one, and attempting to make a left turn. At the intersection, the front of V1 contacted the right side of V2.”
The report says the intersection is signalized but gives no signal state, and each driver blames the other.
Was it avoidable? yes: the SUT’s controller never brakes
Which actors are necessary, alone or together? the motorcycle; not the pedestrian and the pole
What least change avoids it? the motorcycle’s gap, 5 → 20 m
Did a broken traffic rule cause it? not resolved: the files give no signal state
When was it lost? 0.77 s before the contact
Which variables carry it? {SUT speed}, {V2 speed}, {V2 gap}, {V2 speed, V2 gap}
How far can the scenario shrink and still fail? without the pedestrian and the pole: same outcome, 2 of 4 road users kept; it fails only with the motorcycle within 10 m: search space −50 %
National Highway Traffic Safety Administration (NHTSA), National Motor Vehicle Crash Causation Survey (NMVCCS), 2005–2007: US government public data. Case: NMVCCS 2007-03-018, in the NMVCCS case viewer (enter the case number).
A generated scenario, from a plain-words description
An LLM turns a plain-words description into an OpenSCENARIO file and fills in what the text leaves out.
“On a two-lane urban road, ego drives at 40 kph in the right lane. A van is parked on the right edge ahead of ego and hides a cyclist, who pulls out from behind the van into ego’s lane. Variables: the cyclist’s speed between 10 and 20 kph, the van’s distance ahead of ego between 20 and 50 m, ego’s speed between 30 and 50 kph.”
The text makes the parked van the hazard: it hides the cyclist.
Was it avoidable? yes: the SUT’s controller never brakes
Which actors are necessary, alone or together? the cyclist; not the van
↪ the van the text says hides the cyclist
What least change avoids it? the SUT’s speed, 10 km/h lower
Did a broken traffic rule cause it? none: the files declare no traffic rule to break
When was it lost? 0.67 s before the contact
Which variables carry it? {cyclist speed}, {SUT speed} under all KPIs; {van distance} under one KPI only
How far can the scenario shrink and still fail? without the van: same outcome, 2 of 3 road users kept; no failure below 35 km/h: search space −20 %
The text makes the van the hazard; removing it changes no outcome.
Inside your test loop
An engine that rides on your stack — not a stack to migrate to.
YOUR TEST PLATFORM
One tool call
causal_map, through a standard agent-integration protocol (MCP), from a dev environment, a chat surface or a CI step.
Open-standard native
Consumes and emits the open scenario and road-network formats. No proprietary scenario language to adopt or migrate to.
Simulator-agnostic core
A thin adapter over the real simulation tool rather than a reimplementation.
Compute-agnostic
Locally, on-premise or in the cloud, and suited to CI/CD.
It explains failures; it does not certify safety.
Value
What an explanation buys you
Trust
Every finding is measured, not modelled; every case shown ships with the scenario file that reproduces it.
Smaller scenarios
Drop the actors the failure does not need, and the values that never fail: a leaner test, fewer runs every release.
SUT failure or unavoidable contact
Was it avoidable? If not, no ticket.
A target for the fix
When it was lost, and the least change that avoids it, as a scenario you can replay.
What to cover
The carrying variables are the coverage model, written for you in OpenSCENARIO 2.0.
Compare system versions
Same scenario, two systems: what changes in the explanation, what holds.
Engineers stay the experts
AI writes the scenarios; your team still sees why each one fails.
Evidence for ISO 21448 (SOTIF)
Triggering conditions, measured: which road users matter, which variables carry the failure, over which values.
About
Tempero
Tempero (Latin: to temper, to refrain) exists to mitigate the risks of high-risk AI systems. The founding team comes from automated reasoning, formal testing and constraint solving, with prior roles at Microsoft Research and autonomous-systems institutes. Automotive SDVs come first, because they rest on mature standards such as OpenSCENARIO.
From Pass/Fail to a Measured Why
See it for yourself: personal access to the Causal Map server, on request.
Request access