NCEP/EMC Global Model Precipitation Verification

http://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/

Input Data: Ying Lin's G211/RFC FHO VSDB data saved at tempest:/mmb/wd22yl/vsdb.
Monte Carlo Significance Test: Fanglin Yang's presentation
NCEP/EMC MMB QPF: Ying Lin's web

• Definition of contingency table used for computing ETS and Bias Scores:
  • Hits (a): occasions where both forecast and observation are greater than or equal to a threshold;
  • False alarms (b): occasions where forecast is above a threshold whereas observation is under the same threshold;
  • Misses (c): occasions where the observation is above a threshold and forecast is under the same threshold;
  • No forecasts (d): occasions where both forecast and observation are under the threshold.
• Bias Score: BS=(a + b)/(a + c)
  measures over-forecasts (BS>1) or under-forecasts (BS<1) precipitation frequency over an area for a selected threshold.
• Threat Score: TS=a/(a + b + c)
  Measures the fraction of observed and/or forecast events that were correctly predicted. It is sensitive to hits, penalizes both misses and false alarms, does not distinguish source of forecast error, depends on climatological frequency of events (poorer scores for rarer events) since some hits can occur purely due to random chance. TS=1 means a perfect forecast.
• Equitable Threat Score: EQ_TS=(a - ar)/(a + b + c - ar)
  where ar is the expected number of correct forecasts above the threshold in a random forecast where forecast occurrence/non-occurrence is independent from observation/non-observation, ar=(a + b)*(a + c)/(a + b + c + d). EQ_TS=1 means a perfect forecast. EQ_TS <=0 means the forecast is useless.