Skip to main content
Wildfire Insurance → Investigation 06

Does It Hold Across Fires?

A model that works on one fire might be overfit. We ran the same classification pipeline on Kincade and Camp Fire — two different terrains, weather patterns, and outcomes.

2
Fires Cross-Validated
316 + 155
Properties
91% + 88%
Reclassified
The Comparison

Same Pipeline, Different Fires

Kincade burned through Northern Sonoma County wine country in October 2019. Camp Fire leveled the town of Paradise in November 2018. Different terrain, different weather, radically different outcomes. We ran the identical four-level classification pipeline on properties from both fire domains.

Metric Kincade Camp Fire
Properties 316 155
Properties burned 21 (7%) 131 (85%)
L0 classification 96% HIGH 100% HIGH
L3 classification 86% EXTREME, 9% HIGH 87% EXTREME, 12% HIGH
L0 → L3 reclassification 91% 88%
L3 sensitivity 100% 99%
L3 specificity 6% 0%
EAL divergence (L0 vs L3) 133% 128%

EAL = Expected Annual Loss. Divergence = (L3 EAL − L0 EAL) / L0 EAL. Both fires use 200 MC draws.

Reclassification

L0 → L3 Reclassification Side by Side

L0 → L3 Reclassification Rate by Fire

Despite radically different burn outcomes — 7% of Kincade properties burned vs. 85% at Camp Fire — the reclassification rate is nearly identical. The zone-based model fails to differentiate in both environments.

EAL Divergence

How Much the Loss Estimate Changes

EAL Divergence: L0 vs L3 by Fire

L0 understates Expected Annual Loss by 128–133% across both fires. This isn’t a rounding error — it’s more than doubling the loss estimate when the model accounts for terrain, fuel, wind, and uncertainty.

Finding

Not a Single-Fire Artifact

Finding
The pattern holds across both fires. Despite radically different outcomes — 7% of Kincade properties burned vs. 85% at Camp Fire — the fidelity effect is consistent: L0 understates risk by 128–133%, and 88–91% of properties get reclassified. This is not a single-fire artifact.
Context

Why Camp Fire Is Different

Camp Fire destroyed nearly everything in Paradise. Under L0, all 155 properties are HIGH — but that’s actually correct for Camp Fire. The model’s value is that it goes further: 87% are EXTREME (almost certain to burn) while 12% are HIGH (probable but not certain). Even in a catastrophe, the fire model differentiates the last few percent.

Camp Fire’s 0% specificity makes sense: when 85% of properties burn, there are almost no true negatives to identify. The model isn’t wrong — the fire really did burn everything. What matters is the differentiation within the burn zone.

Consistency

The Structural Effect of Fidelity

Both fires show the same qualitative pattern:

  1. L0 lumps properties into one tier (no differentiation)
  2. L1 adds the EXTREME tier (screening catches the worst)
  3. L2/L3 redistribute within HIGH/EXTREME (simulation refines the picture)

The reclassification rate is stable at 88–91%. The EAL divergence is stable at 128–133%. These are not artifacts — they’re the structural effect of fidelity.

The classification thresholds and loss rates are identical across both fires. The consistency comes from the fire model’s behavior, not from parameter tuning.