Ongoing research into advanced weather prediction using AI-driven NEXRAD radar analysis.
Generated by AI system Zeus
Moore, OK EF-5 Tornado — May 20, 2013 — V3 U-Net prediction overlay
KMAF Tornado (Test Set) — May 3, 2024 — V3 U-Net correctly detected
| Model | Separation | Recall | FP Rate | Accuracy | Status |
|---|---|---|---|---|---|
| V1 | 11.6x | 40% | 20% | — | ✅ Baseline (small dataset) |
| V2 | 1.2x | 100% | 80% | — | ❌ Binary masks failed |
| V3 | 3.19x | 40% | 0% | 70% |
Key finding: The physics masks contain no discriminative signal between tornado and non-tornado storms at single-scan resolution. The mask generator detects rotation zones, hook echoes, and BWERs in both tornadic and non-tornadic storms with nearly identical statistics.
Configuration:
- Architecture: U-Net (3 in, 1 out), 25.3M parameters
- Loss: MSE(0.35) + Dice(0.45) + TV(0.05) + Focal(0.15) + AntiZero(0.50)
- Training: Phase 1 only, 15 epochs, ~90 balanced events per class
- Weighted masks: Tornado 2.0×, Non-tornado 0.05×, scaled ×3.5 in training
- Checkpoint:
experiments/physics_segmentation_training/best_model_v3.pth
Validation performance: 4.02x separation ratio on validation split.
| Metric | Tornado | Non-tornado | Separation |
|---|---|---|---|
| Max range | 0.008–0.241 | 0.008–0.201 | 1.21x |
| Mean range | 0.0006–0.0015 | 0.0003–0.0012 | 1.26x |
The masks are statistically identical for both classes. The physics mask generator detects:
- Velocity couplets (rotation zones)
- Hook echoes (reflectivity patterns)
- BWERs (updraft indicators)
These signatures appear in both tornadic and non-tornadic supercells. A mesocyclone looks identical to the mask generator whether it produces a tornado or not.
The NOAA NCEI Storm Events database provides 6,142 verified tornadoes with:
- GPS coordinates (lat/lon)
- EF ratings (EF0–EF5)
- Timestamps (UTC)
- State and location names
The labels are correct. The problem is the masks, not the labels.
Sample: 5 tornado + 5 non-tornado events from temporal holdout (2023–2025)
| Event | Label | Pred Max | Scans>0.50 | Consecutive>0.50 |
|---|---|---|---|---|
| 🌪️ KMAF 2024-05-03 | Tornado | 0.979 | 19 | 13 ✅ |
| 🌪️ KBBX 2025-05-19 | Tornado | 0.000 | 0 | 0 ❌ |
| 🌪️ KTBW 2023-12-09 | Tornado | 0.000 | 0 | 0 ❌ |
| 🌪️ KPAH 2024-07-09 | Tornado | 1.000 | 22 | 22 ✅ |
| 🌪️ KLTX 2025-05-17 | Tornado | 0.000 | 0 | 0 ❌ |
| KEWX 2023-07-15 | Non-tornado | 0.000 | 0 | 0 ✅ |
| KDFX 2023-07-07 | Non-tornado | 0.000 | 0 | 0 ✅ |
| KLWX 2023-10-14 | Non-tornado | 0.621 | 2 | 1 ✅ |
| KATX 2023-07-16 | Non-tornado | 0.000 | 0 | 0 ✅ |
| KGYX 2025-04-18 | Non-tornado | 0.000 | 0 | 0 ✅ |
| Metric | Single Scan | Temporal (≥3 scans) |
|---|---|---|
| Tornado Detected | 2/5 (40%) | 2/5 (40%) |
| Non-tornado Flagged | 1/5 (20%) | 0/5 (0%) |
| Precision | 0.667 | 1.000 |
| Recall | 0.400 | 0.400 |
| Accuracy | 60% | 70% |
The model is bimodal: it either strongly detects (KMAF: 0.979, KPAH: 1.000) or completely misses (KBBX, KTBW, KLTX: 0.000). The missed tornadoes have weaker physics signals:
| Event | Physics Mask Max | After 2.0× Weight | After ×3.5 Scaling |
|---|---|---|---|
| KBBX (missed) | 0.13–0.16 | 0.26–0.32 | 0.91–1.12 |
| KTBW (missed) | 0.13–0.15 | 0.26–0.30 | 0.91–1.05 |
| KLTX (missed) | 0.09–0.14 | 0.18–0.28 | 0.63–0.98 |
| KMAF (detected) | 0.13–0.21 | 0.26–0.42 | 0.91–1.47 |
| KPAH (detected) | 0.16–0.22 | 0.32–0.44 | 1.12–1.54 |
The missed tornadoes' signals fall right at the model's decision boundary. The physics masks don't encode enough information to distinguish them from non-tornado storms.
Single-scan U-Net segmentation has hit its data-imposed ceiling. The recommended approach:
The CNN already achieves good classification accuracy. Use it as the primary detector.
Require ≥3 consecutive scans with pred_max > 0.50 to confirm a detection. This eliminates single-scan false positives.
Use the existing couplet detector as a secondary confirmation:
velocity_difference > 40 m/s→ STRONG rotationvelocity_difference > 20 m/s→ MODERATE rotation
Combine CNN probability + temporal persistence + couplet strength into a single tornado probability using gradient boosting or a simple weighted score.
| Resource | Path |
|---|---|
| V3 Training | scripts/training/train_unet_v3.py |
| Two-Phase Training | scripts/training/train_unet_two_phase.py |
| Mixed Loss Training | scripts/training/train_unet_mixed.py |
| CNN Training | scripts/training/train_cnn_model.py |
| Temporal Training | scripts/training/train_temporal_end_to_end.py |
| Resource | Path |
|---|---|
| Full U-Net Validation | scripts/validation/validate_full_unet.py |
| Physics U-Net Validation | scripts/validation/validate_physics_unet.py |
| Test Split Validation | scripts/validation/validate_test_split.py |
| Pipeline Validation | scripts/validation/validate_pipeline.py |
| Resource | Path |
|---|---|
| Batch Segmentation | scripts/batch/batch_segmentation.py |
| Temporal Aggregation | scripts/batch/temporal_aggregation.py |
| Temporal Fusion | scripts/batch/temporal_fusion.py |
| Resource | Path |
|---|---|
| Moore EF-5 GIF Generator | scripts/visualization/generate_moore_gif.py |
| Heatmap Animations | scripts/visualization/generate_heatmap_animations.py |
| Fusion Map | scripts/visualization/visualize_fusion_map.py |
| Heatmap Evaluation | scripts/visualization/evaluate_unet_heatmaps.py |
| U-Net Attention Maps | scripts/visualization/attention_maps_unet.py |
| Resource | Path |
|---|---|
| Mixed Loss Functions | tornado_detection/segmentation/spatial_loss.py |
| U-Net Model | tornado_detection/segmentation/unet_model.py |
| Dataset Loader | tornado_detection/segmentation/training_dataset.py |
| Physics Mask Generator | tornado_detection/segmentation/physics_mask_generator.py |
| Storm Tracker | tornado_detection/segmentation/storm_tracker.py |
| Standard Loader | tornado_detection/pipeline/standard_loader.py |
| Raster Converter | tornado_detection/pipeline/raster_converter.py |
| Resource | Path |
|---|---|
| Weighted Masks | data/nexrad/weighted_masks/ |
| Physics Masks | data/nexrad/physics_masks/ |
| Split Manifest | data/nexrad/split_manifest.json |
| Event Manifest | data/nexrad/event_manifest.json |
| NOAA Tornadoes | data/nexrad/storm_events/noaa_tornadoes.json |
| Resource | Path |
|---|---|
| Moore EF-5 GIF | media/moore_ef5_v3_prediction.gif |
- Physics masks can't distinguish tornado from non-tornado at single-scan resolution — The signatures (couplets, hooks, BWERs) appear in both classes
- Weighted continuous masks don't help — Scaling targets doesn't create signal that doesn't exist
- Anti-zero loss prevents collapse — Essential for preventing the model from predicting all zeros
- Temporal filtering eliminates FPs — Requiring ≥3 consecutive scans above threshold gives 0% FP rate
- The CNN is still the best single-scan detector — Focus on cascading, not segmentation
Generated: May 29, 2026