Bridging the gap

Motivations

When applying traditional pointwise verification methods to spatial weather fields, forecasts that are nearly correct but slightly displaced in space can be unfairly penalized, counting both a missed event and a false alarm for the same feature. This phenomenon is known as the double penalty effect. To address this, as well as to gain more in-depth understanding in the nature of the forecast error, specialized spatial verification methods have been developed. These methods are designed to account for spatial structure, displacement, and scale, providing a more meaningful assessment of forecast performance for spatial fields.

Similarly, traditional verification scores often can be synthetically improved by slightly modifying the forecast, disregarding the forecaster’s initial belief. This phenomenon is known as hedging. Examples include categorical scores which can be improved by over-forecasting. Consequently, motivated by theoretical decision-making, consistent/proper scores have been developed as improved verification methods resilient to hedging. Despite of the common interest behind these two approaches, the explicit links between these apparently diverging approaches have not been fully explored. The spatial verification tools community has primarily focused on developing interpretable methods resilient to the double penalty effect, often at the expense of theoretical guarantees. In contrast, the consistent/proper scores community has emphasized ensuring theoretical properties, sometimes overlooking interpretability.

This research project will address the following questions. What are the links between the two approaches and how can we combine them to improve spatial verification? How to properly extend spatial verification methods to ensemble forecasts? Does the double penalty effect apply to ensemble forecasts? Does the verification of AI-based forecasts require new methods? What are desirable properties of spatial verification methods for ensemble forecasts and how to check them?

Goals

1. 1
   Encourage and coordinate exchanges between the two communities
2. 2
   Understand and explicit the gaps and links between the two approaches
3. 3
   Develop new proper methods for spatial verification of ensemble forecasts
4. 4
   Facilitate meta-verification
5. 5
   Focus on some specific tailored applications

Participants

The research project gathers 57 participants.

Meteorological centers: SMN , BoM , GeoSphere , ECCC , ECMWF , Météo-France , DWD , NCMRWF , KNMI , MetNorway , MeteoSwiss , NOAA .

Academic institutions: Laval University ; CNRS, Marie and Louis Pasteur University ; University of Bonn, University of Duisburg-Essen, HITS, KIT, Marburg University ; University of Ljubljana ; Chalmers University of Technology ; University of Geneva, ETH Zurich ; University of Bath, University of Exeter, University of Oxford, University of Reading ; Cornell University .