Skip to content

CAL: Calibrate reflectivity coverage thresholds in codebook#32

Merged
rcjackson merged 1 commit into
ARM-Development:mainfrom
rcjackson:calibrate-reflectivity-thresholds
Jul 13, 2026
Merged

CAL: Calibrate reflectivity coverage thresholds in codebook#32
rcjackson merged 1 commit into
ARM-Development:mainfrom
rcjackson:calibrate-reflectivity-thresholds

Conversation

@rcjackson

Copy link
Copy Markdown
Collaborator

Summary

Recalibrates the three quantitative reflectivity-coverage reclassification thresholds in CODEBOOK.md (Section 3.1) against 1,212 hand-labelled CSAPR2 scenes (Bankhead National Forest). These thresholds drive the automatic label overrides applied by criteria_from_codebook / apply_criteria_to_labels.

Rule (source → reclassify target) Field Old New Basis
No Precipitation → Isolated Convection pct_gates_50dbz 0.002% 0.005% Old value overrode ~3% of genuine No-Precip scenes (observed max 0.0045%). pct_gates_50dbz does not separate these two classes, so this rule only guards against gross mislabels; raised just above the full observed No-Precip envelope.
Stratiform → Mesoscale Convective System pct_gates_50dbz 0.02% 0.035% Youden-optimal cut vs MCS (J=0.72). False overrides of genuine stratiform drop 13% → 6%, still catching ~79% of MCS.
Isolated Convection → Mesoscale Convective System pct_gates_30dbz 1.0% 1.3% Youden-optimal cut vs MCS (J=0.70). False overrides 16% → 13% at equivalent catch.

Method

For the two rules where the source and target classes genuinely separate on the field (Stratiform→MCS, Isolated→MCS), thresholds are set at the Youden-J optimal cut (maximizing true catch − false override). For No-Precip→Isolated Convection the field does not discriminate the classes (Isolated Convection's median 50 dBZ coverage ≈ 0), so the threshold is set just above the observed No-Precip envelope to avoid overriding genuine No-Precip scenes.

Verification

  • criteria_from_codebook('CODEBOOK.md') parses all three new thresholds with correct reclassify targets.
  • Applying old vs. new criteria to the ground-truth labels via apply_criteria_to_labels: spurious hard-overrides of correctly-labelled scenes drop from 82 (6.8%) → 43 (3.5%).
  • Added a v1.1 changelog entry.

🤖 Generated with Claude Code

Recalibrated the three quantitative reflectivity-coverage reclassification
thresholds in CODEBOOK.md against 1,212 hand-labelled CSAPR2 scenes:

- No Precipitation  pct_gates_50dBZ  0.002 -> 0.005 percent
  (old value overrode ~3% of genuine No-Precip scenes; 0.005 clears the
   full observed No-Precip envelope, max 0.0045)
- Stratiform Precipitation  pct_gates_50dBZ  0.02 -> 0.035 percent
  (Youden-optimal cut vs MCS, J=0.72; false overrides 13% -> 6%)
- Isolated Convection  pct_gates_30dBZ  1.0 -> 1.3 percent
  (Youden-optimal cut vs MCS, J=0.70; false overrides 16% -> 13%)

Net effect: spurious hard-overrides of correctly-labelled scenes drop
from 6.8% to 3.5% while catch of genuine mislabels is preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rcjackson rcjackson merged commit 00042a4 into ARM-Development:main Jul 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant