California Freight Cleanup → Investigation M-5
How much does upstream uncertainty move the final portfolio recommendation?
Primary shift: Δ P(opt) = −0.0070 (−1.09%) at Investigation 6-3 σ×2Investigation 3-6 and Investigation 3-10 decompose cascade uncertainty at the input-prior level (which input driver explains the most NB variance?). Investigation M-5 addresses the complementary question: if a per-investigation sigma is doubled at some upstream node, how much does it move the headline P(optimal) at the end of the chain? The framework is the deliverable—any future “what if Inv X’s sigma were doubled?” question for any of the six covered investigations can be answered in under one second.
The decision
The California Freight Cleanup cascade chains dozens of investigations together. Each one carries its own uncertainty envelope, which gets baked into its results and passed downstream as a point estimate. There is no native machinery for the question a careful reviewer will always ask:
“If the uncertainty in one of your upstream inputs were twice as wide, how much would it change the portfolio recommendation at the end of the chain?”
Investigation M-5 builds that machinery for the six most critical upstream inputs and demonstrates it with four stress tests, each doubling the uncertainty at one node.
Framework architecture
The framework has three components, implemented in
rfaq.shared.uncertainty_propagation:
- PerturbationSpec dataclass—declares which upstream investigation, which dotted field path, what kernel (multiplicative / additive / lognormal / sigma_scale), and what scale factor.
-
Adapter registry—six registered investigations, each
exposing a function that reads its own
latest/results.json, optionally applies the perturbation, and returns a dict of (N_MC,) numpy arrays representing its MC-eligible outputs. - propagate_through_cascade(spec, terminal_qoi, n_mc, seed)—runs the chain (Investigation 6-3 → 5 → 6 → 15 → 23 → 44) twice: once with no perturbation (baseline), once with the spec applied at its target investigation. Reports baseline → perturbed shift with bootstrap CIs.
Demonstration results
| Perturbation | Kernel | Scale | Δ P(opt) | Δ mean NB ($B) |
|---|---|---|---|---|
| Investigation 6-3 β posterior σ (primary) | sigma_scale | ×2 | −0.0070 (−1.09%) | −$0.237 |
| Investigation 6-4 cost_cv | multiplicative | ×2 | −0.0015 | −$0.247 |
| Investigation M-1 D deaths_avoided σ | sigma_scale | ×2 | +0.0015 | −$0.247 |
| Investigation 6-6 emissions_prior sigma_log | sigma_scale | ×2 | +0.0015 | +$1.547 |
Framework baseline P(opt) for D_all_in_4B is ~0.644, which differs from Investigation 6-6’s reported 1.000 because the framework uses the canonical-wide VSL ($5M–$20M) and a 6-adapter chain while Investigation 6-6 uses the EPA-narrow band ($7.4M–$13.4M) and its own internal MC engine. The load-bearing outputs are the shifts, not the baseline level.
Findings
Doubling the health-risk uncertainty moves the portfolio recommendation by −1.09% — consistent with the sensitivity analysis from a different angle
Investigation 3-10 attributes ~15% of cascade NB variance to βPM2.5. A 2× sigma inflation roughly quadruples the variance contribution (variance scales as σ2), so the expected NB-variance shift is on the order of 4× the baseline 15% — broadly consistent with the observed P(opt) sensitivity. Investigation M-5’s independent methodology (sigma propagation rather than Saltelli sampling, on the same cascade chain) produces a headline shift of −1.09%—consistent with a 15% variance share. The two investigations now provide independent verification of the same underlying sensitivity via different analytical routes.
No single-input stress test comes close to reversing the portfolio recommendation
Across all four sigma-doubling stress tests (Investigation 6-3, 15, 23, 44 each scaled ×2), the largest absolute shift in P(opt) is 0.0070. The 0.05 threshold—a 5 percentage-point absolute shift in the probability of D being the NPV-argmax—is not approached by any covered perturbation. The headline recommendation is robust to plausible sigma-doubling stress within the six covered upstream nodes.
The framework is the real deliverable: any covered “what if uncertainty doubled?” question runs in under a second
Per-question marginal wall-clock cost at N_MC = 4,000 is approximately 0.4 seconds. Any future reviewer question of the form “what if Inv X’s sigma were doubled?” can be answered on demand for any of the six covered investigations. This is the operational value: not the specific numerical answers in this run, but the framework that makes future answers available without re-running the full cascade.
Six of 57 inputs are covered (10.5%) — honest about what the framework can and cannot claim
6 of 57 investigations (10.5%) currently expose adapters. The 51 uncovered include Investigation M-2 (PBVI cost-overrun HalfNormal), Investigation 8-2 (adaptive monitoring information gain), and Inv 30 (PCE wildfire ignition)—all carrying heavy-tailed envelopes the linear-Gaussian kernel cannot represent. The correct claim: “robust to sigma-doubling at every covered upstream node (Investigation 1-1, 6, 15, 21, 23, 44).” Not “robust across the full cascade.”
Coverage map
| Investigation | Adapter? | Perturbable field |
|---|---|---|
| 21 (CRF posterior) | Yes | l3_hierarchical.mu_posterior_sigma |
| 5 (transport scenarios) | Yes | per-scenario deaths_avoided MC envelope |
| 6 (building scenarios) | Yes | per-scenario deaths_avoided MC envelope |
| 15 (portfolio frontier) | Yes | frontier_band.named_portfolio_band.D_all_in_4B.deaths_avoided |
| 23 (robust optimization) | Yes | uncertainty_envelope.cost_cv |
| 44 (CRF-conditional decision) | Yes | posterior_spec.emissions_prior.sigma_log |
| 22, 27, 29, 30… (51 others) | No | Adapter expansion deferred (Phases A–C) |
Caveats
- Partial coverage is the central constraint. 6 of 57 investigations
(10.5%). The
coverage_reportfunction lists all 51 uncovered investigations. No “robust across the full cascade” claim is defensible at this coverage level. - Linearity of sigma propagation. Each adapter assumes downstream response is approximately linear in the perturbed sigma. For nonlinear chains (Investigation M-2 POMDP belief-state, chaotic dynamics) the framework understates true sensitivity. Acceptable for sigma-doubling; not for sigma-decimation.
- No cross-investigation correlation. The production the cascade's joint sampler encodes corr(βPM2.5, βO3) = 0.3 and corr(emissions, surrogate_σ) = 0.4. These couplings do not propagate through the framework chain. Full correlated propagation requires Kucherenko’s method (Phase C expansion).
- Baseline-level mismatch with Investigation 6-6. Framework baseline P(opt) ~0.645 vs Investigation 6-6’s 1.000 is expected and documented. Quote only the shifts, never the absolute baseline from this investigation.
- Hand-written adapters drift silently if upstream investigations rename schema fields. Adapter audits must be done manually after any structural upstream re-run.
Provenance
| Item | SHA-256 (12-char) | |
|---|---|---|
| results.json | 453a980f5b37 |
|
| analysis.md | — | |
| scenario.md | — | |
| Framework module | rfaq.shared.uncertainty_propagation |
|
| Upstream: Investigation 6-6 (baseline P(opt)) | investigations/44_crf-conditional-decision/latest/results.json | 5ce9bcd8b87b |
| Integration test | tests/test_inv61.py::TestInv61 — asserts |Δ P(opt)| < 0.05 |
|
| Run timestamp | 2026-05-04T17:03:58 N_MC = 4000 per pass seed = 20260503 | |