Philippines Typhoon: Anatomy of a Forecast Bust
UPDATE JUNE 25TH: AccuWeather.com has produced a graphic showing this event:
ORIGINAL REPORT:
There was (what some in the media may refer to as) a spectacularly inaccurate forecast this week from the Computer Forecast Models [JessePedia], and that was that Typhoon Fengshen (Frank) would miss the Philippines. Thus far, no one in the media is really talking about this, but they should. To help them stay on task, I offer the following facts. Here is where we stand this morning, according to our World News Story (PREMIUM | PRO)the storm having made direct landfall on the Philippine islands.
On Wednesday at 5 PM, we issued this map in an electronic press release and (I assume) on AccuWeather.com as a Weather Headlines (PREMIUM | PRO):
The press release stated:
This was all based on what is generally referred to as the "Navy" or "JTWC (Joint Typhoon Warning Center" forecast, which (I think) usually does a good job forecaster international hurricanes, and which we would normally have no reason to not believe. Their map at 7 PM Wednesday still looked like this (presumably* it was further east before that).
The forecasts kept "trending westward" (as AccuWeather.com meteorologist Jim Andrews noted in his blog yesterday (PREMIUM | PRO)) until the storm was finally progged to hit land yesterday evening (though forecasts continued to trend westward overnight).
SO HOW FAR OFF WAS THE FORECAST?
Who among us knows the geography of the Philippines as much as the hurricane-prone regions of the United States. Certainly I do not. So I used Google Earth to help me find out. The Wednesday evening forecast appeared to be off by about 4 degrees of longitude.
Comparing the Philippines to Florida, that would be the equivalent of placing the storm well off the coast, versus onshore 36 hours in advance (or placing it in the Gulf vs. in the Atlantic at a particular time).
I don't recall off the top of my head how many U.S. hurricane forecasts have been that far off 36 hours in the future, but I'm sure it has happened. Probably not a lot - although models bounce hurricanes around quite a bit in the long-range forecasts, they usually have it more or less pinned down by 36 hours out (which is good because evacuations can take that long). It's possible that the lack of high-resolution models in the Pacific led to this bad forecast. It's also possible that the JTWC forecast favored a particular model, such as the GFS, or their own Navy model (the NOGAPS) and they stuck with it until the bitter end (both were still forecasting non-landfalls yesterday morning, click here to download movies of their forecasts from 12Z yesterday and 00Z today).
WHY WAS THE FORECAST SO FAR OFF?
Because I don't forecast for that part of the Earth, I don't have any specific insights as to what went wrong here. But as I've said before, hurricane/typhoon prediction (and really, the prediction of any low pressure system) depends on how it gets caught in between the cogs (circulations) of other high and low pressure systems. If one of those is weaker or stronger than the models thought, it can make a big difference in the track of the storm (ever seen "Plinko" on the Price Is Right? Try that, but have the size of the pegs dynamically change.)
WHAT DO YOU THINK?
Is this a huge bust or normal forecast variability? Would the forecast have been better with better models? Should the Navy have acted faster on the westward model swing? Should we have been more independent from their forecast? Post a Comment below to let me know. I hope you appreciate me bringing this forecast error into the light and not trying to hide it -- and I'm not blaming the Navy, they were using the same models that we all have.
SIDENOTES:
1. Conspiracy theorists rejoice. The JTWC issued two forecasts before 7 PM Wednesday but they (and also the 5th forecast) are marked "Forbidden; You Do Not Have Permission To Access" on the the Naval Research Laboratory tropical weather website's archive of forecast tracks. (Coming from an IT background, however, I can tell you that particular error message is generally a technical problem and not typically used to hide web files).
Because it has the official JTWC stamp, it is generally believed that the track map shown above is a human-produced forecast, but I have no way of confirming that as of this writing (it wasn't in their FAQ). It is not actually "The Navy" by the way; their site states that the JTWC is "the U.S. Department of Defense agency responsible for issuing tropical cyclone warnings for the Pacific and Indian Oceans" and is "manned by 26 U.S. Air Force and Navy personnel."
I find it curious that the Naval Research Laboratory tropical weather website labels the images from this forecast "ATCF," a conglomeration of models that I also call the Model Spread [JessePedia]. The JTWC site says nothing about the ATCF, saying that "As of December 2006, the multi-model ensemble called "Consensus-W" was comprised of 11 members: NOGAPS, GFDN, COAMPS, AFWA MM5, NCEP GFS, WBAR, UKMET, JGSM, JTYM, TCLAPS, and ECMWF." It could just be a naming misshap.
Comments (3):
joe:
It's another unfortunate disaster and as I went through the comments of many, there's quite a bit
of people putting a finger on PAGASA. Who else?...you might add. During elections, candidates very seldomly tells you what he/she can do for the country, but rather, spits out bad
informations/publicity about the competitor. I'm
not saying that we give PAGASA a feather on the hat, but let's try to find a SOLUTION. I'm
convinced that the whole PAGASA is doing their
best to give the public what we deserve based on their limited equipments, capability, training or
whatever. It would be unfair for us to make judgement, jokes, or embarrasing comments which
gives low morale for the hard working PAGASA
employees. Additionally, it's unwise for us to
make judgemental statements with our very minimal
knowledge about weather patterns, climatology,
Meteorology and the like. We don't even know if there's politics/communication problem between Japan, Guam, or Taiwanese Weather Bureau. There's
a lot of factors involved which is transparent from us. Now, the Sulpicio tragedy is another
story. We don't know who actually erred and I'm not in the position to blame someone beacuse I don't know the whole scenario. Let's just pray for all the victims that may God bring them to
Eternal Life. For the victim's families, may God give them strength that in these time of their sorrow, may he bring consolation, understanding,
love, and peace in their hearts.
Thank you.
Posted by joe | June 28, 2008 2:09 PM
Kathleen Reif:
I think you bring up an extremely important point, one that as a Florida resident, I've noticed before, as in Hurricane Mitch in '98.
NHC models took the monster Cat 5 north of the Bay Islands, predicting that it would turn northwest. Nope. Mitch did a 180 south not north
Posted by Kathleen Reif | June 22, 2008 11:35 PM
Mark Warriner:
VERY good article by Jesse Ferrell,excellent investigative journalism, including the sidenotes.
This situation needs to be brought to light as it is not receiving the attention it deserves. This was a catastrophic misforecast and many people have died because of it.
Posted by Mark Warriner | June 22, 2008 11:08 AM
