DOWNLOADS
AMS Competition Poster
Climatology Data
Forecaster Details (example)
Forecast Details (csv)
R Code


RPI2.0 Seasonal Forecast Competition — Results

We are pleased to announce the completion of our 2013 Seasonal Landfall Forecast Competition.

The scoring for each landfall forecast submission is conducted via the Forecast Information index:

FIj=log(pf,o,j)-log(pc,o,j),

Where pf,o,j is the forecast probability (f) assigned to the observation (o) of the predictand (j). The Forecast Information score is computed for each submission, and then summed over all categories, yielding a final score, which we present in the results below.

Higher ‘Forecast Information’ scores are given for predicted probabilities of landfall that were higher than climatological probabilities, for regions in which landfalls were actually recorded. A schematic example of a forecast scoring scenario is outlined here:

An increasingly positive FIj indicates greater skill above climatology, for a given count of landfalls in the region examined. A negative FIj indicates no skill above climatological probability of landfall. More details about this scoring methodology are available on our website.

In accordance with National Hurricane Center archives for the 2013 season, this particular year saw only one tropical cyclone make landfall, Tropical Storm Andrea on 6 June, on the northwest coast of Florida. Details of that storm are outlined here:

http://www.nhc.noaa.gov/data/tcr/AL012013_Andrea.pdf

This one event constitutes landfalls in the following pre-defined landfall regions, for the purposes of the competition:

- US
- Gulf Coast
- Florida

Details about each winner’s submission are available in the table below (information is based on the time of submission):

 December 2012March 2013May 2013August 2013
MembershipAGU, OtherNoneAMSAGU, AMS
Highest level of EducationPh.D.BachelorsMastersMasters
EmploymentAcademiaAcademiaPrivate SectorAcademia
Employment PositionResearcherResearcherRisk ModelerGraduate Student
Forecast BasisOtherOtherOtherStatistical Model
Forecast Information Score4.082.992.591.10

Overall the Crowd’s Forecast Information score was negative in each round of the competition, indicating no skill against climatology. More details about each round of the competition are available on the RPI website here: December 2012, March 2013, May 2013 and August 2013.

Generally speaking, there were too few submissions in each category for a meaningful and statistically significant crowd forecast outcome. There is currently no intent for RPI to run another competition internally in 2014.