Earthquake Model Training Results
This page presents a comprehensive evaluation of the earthquake damage classification models developed within the MAIPARK Vulnerability Analyzer framework. The models were trained using historical insurance data, consisting of both cession and claim records.
The dataset represents observed building performance under past earthquake events and forms the empirical basis for learning structural damage behavior across different vulnerability conditions. After data cleaning and validation, a total of 56,795 historical building records were used in the analysis.
The full dataset was randomly divided into:
- 80% Training Data — used to train the Random Forest models and learn damage classification patterns.
- 20% Testing Data — held out for independent performance evaluation to ensure generalization capability.
Two seismic intensity measures are evaluated: MMI (Modified Mercalli Intensity) and PGA (Peak Ground Acceleration). Both models predict ordered damage states: None → Slight → Moderate → Extensive → Complete.
MMI-Based Model Performance
The MMI-based model leverages macroseismic intensity to capture human-perceived shaking effects. This representation is particularly valuable for regional-scale vulnerability assessment, where instrumental ground motion data may be sparse or uncertain.
| Metric | Value |
|---|---|
| Accuracy | 0.95 |
| Macro Precision | 0.449 |
| Macro Recall | 0.393 |
| Macro F1-score | 0.417 |
| Weighted F1-score | 0.946 |
| Ordinal MSE | 1.402 |
| Spearman Correlation | 0.528 |
Metric Interpretation
- Accuracy reflects the overall proportion of correctly classified damage states.
- Macro-averaged metrics treat all damage states equally, highlighting model performance on rarer severe damage classes.
- Ordinal MSE penalizes large misclassification jumps more heavily than near-miss errors.
PGA-Based Model Performance
The PGA-based model relies on instrumentally measured ground motion, providing a physics-driven representation of seismic demand. This approach improves sensitivity to higher damage states, particularly under strong shaking conditions.
| Metric | Value |
|---|---|
| Accuracy | 0.954 |
| Macro Precision | 0.476 |
| Macro Recall | 0.404 |
| Macro F1-score | 0.434 |
| Weighted F1-score | 0.95 |
| Ordinal MSE | 1.377 |
| Spearman Correlation | 0.583 |
Key Observations
- Higher Spearman correlation indicates stronger monotonic agreement between predicted and observed damage severity.
- Lower Ordinal MSE reflects fewer severe misclassification jumps.
