IBM Acquires Weather Company IT: Into the Data Unknown

The recent IBM acquisition of much of The Weather Company (TWC) – effectively, everything but The Weather Channel – is an odd duck.  I can’t remember anything quite like it in the computer industry before now.  Indeed, I suspect that IBM does not yet fully understand where in the search for better analytics via Big Data this new addition will take it.  To put it another way, IBM is stepping into an area where the ultimate use of the acquired data is to a surprising extent unknown.

The Potential Benefits of Launching Into the Data Unknown

As I noted in a previous piece about the initial IBM-TWC partnership, some core benefits are easy to see, and the acquisition simply extends them.  The infusion of analytics with weather data allows fine-tuning of customer and supplier behavior analysis and prediction – if it’s raining cats and dogs, the retail store may see more or less traffic than on a sunny day, and extreme weather will inevitably lead to decreased sales and deliveries.  Of equal long-term potential is the use of climate change science to avoid the “if this goes on” approach to weather forecasting that becomes increasingly unable to anticipate both secular year-round warming and increased occurrence of extreme events.  By taking over these functions, IBM allows its customers to drive weather-data acquisition as well as analytics in directions that a TWC-owned platform would likely not have done.

However, according to its presentation of the acquisition, IBM also views the TWC platform as a proven approach for leveraging Big Data in general for new insights, extending to the sensor-driven Web aka the Internet of Things (IoT) the innovative “cognitive” insights of IBM Watson.  I would argue, in fact that the TWC IT acquisition adds less – and more.  Less, because TWC is not on the cutting edge of the Fast Data “new technology wave” that I have written about previously.  More, because if and when it is combined with the raw “real-time sensor” data that governments collect about the weather and related location information (e.g., habitations, flora, fauna, and topography), it provides a simple “location state” of the individual customer or car or delivery truck that can allow as-yet-undefined risk reduction (avoiding the high-water spots in the road), “state of mind” assessment for selling purposes, and  services (rerouting based on likely washouts).

An Odd Way to Slice a Duck

Certain parts of the TWC acquisition put IBM effectively into areas that are not necessarily a good long-term fit – goodness of fit will depend on IBM’s execution of the acquisition. 

For one thing, acquiring the TWC web sites makes IBM effectively a media company.  Hurricane Sandy, in particular, created large followings for the blogs of certain weather forecasters.  While IBM can pretend that these are merely avocations for TWC personnel focused on The Weather Channel, the fact is that they represent an important news and analysis source for a significant segment of the public.  Whatever IBM does with the link between TWC content suppliers and the web sites will indicate whether IBM is going to run the media part of its slice of TWC into the ground or allow the weather forecasters and analysts that remain with TWC to continue leveraging their clout as experts into eyeballs and advertising revenue. 

And the question of who goes where brings up another area where IBM is venturing into new and not necessarily compatible territory:  weather and climate expertise.  As a recent IBM presentation citing weather data’s usefulness to insurance companies shows, applications to mobile car users that can reduce customer risks (avoiding or sitting out a storm) certainly have an upside to them, but what is not clear is whether the insurance company can use such a mobile app frequently enough to reduce overall customer risk substantially (and thereby increase profit margins). 

An obvious case where weather warnings may have an impact on customer risk profiles is a “black swan”, a seemingly unlikely event such as Hurricane Sandy.  However, in that case individual customer advice is likely to have only minor effect – rather, the weather experts must recognize the need for and drive proactive, outside-the-company warnings that ensure lots of customers take care.  TWC has such experts – but where is the place for them in IBM’s new weather-related organization?  Perhaps Watson will have such “domain knowledge” in the future; but it does not have it today, and so “weather analytics” badly needs such expertise.  And then weather analytics needs climate domain knowledge and climate change expertise, else predictive analytics applied to raw weather data will be far too frequently wrong.

The User Bottom Line:  Going Into the Data Unknown Is Good

I have dwelt on my concerns about IBM’s ability to get the most out of its new oddly sliced duck.  The fact remains, however, that, like Watson, a massive chunk of weather data is important not for its immediate applications, but for the ways in which users will drive new types of insights with it.  Just as Watson has spawned new “cognitive” ways to analyze data, driven by user use cases, so the early adopters of weather data will generate use cases that will suggest other applications that we cannot foresee.  The savvy user, therefore, will pick the right time and the right use case to begin to use weather data – and IBM’s TWC capabilities are an excellent place to start.

And if IBM figures out how to leverage TWC effectively in the areas that I have cited, so much the better for the user.  That the weather data will also lead IBM to get even more serious about climate change, I suppose, is too much to hope.  But hope I will.

IBM-Weather Company Alliance: The Benefits and Limitations of Weather-Driven Analytics

Make no mistake about it, combining IBM and The Weather Company (TWC) is about as good as it gets for companies looking to improve their responsiveness to today’s extremes of weather.  This is not just because IBM’s Watson Analytics promises to add in-depth business insights and additional predictability to your basic average weather forecast.  In fact, The Weather Company is if anything the more important partner, because it has established a solid track record for anticipating some of the more unusual effects of such events as Hurricane Sandy – and Jeff Masters of Weather Underground is an effective analyst with a good appreciation of the effects of climate change on weather patterns.

And yet, the recent announcement of the alliance also serves to show how today’s business analytics is severely handicapped in its ability to incorporate likely future changes in climate and their effect on business for all but the next few days.  To put it another way, IBM/TWC can help a Boston-based firm anticipate and adjust on the fly to a 2-foot snow 5 days from now; but to make the necessary repairs to buildings to avoid roof collapses and leaks, and to make plans to truck snow to more remote locations because local “snow farms” can’t handle so much snow, when there has never been a need before – well, that IBM/TWC probably won’t help with.   However, increasingly, that’s where much of the additional weather-related business cost these days resides.  (And of course, what goes for IBM/TWC goes for other analytics solutions in spades)

The real problem here is the lack of a global climate model that operates on the basis of the most likely climate changes over the next 30 years, not just the ones that can be verified by conservative scientific analysis.  To take one example, we know that melting permafrost is likely to have a significant effect in warming the globe by releasing carbon and methane over the next 30 years, but since we can’t rule out the possibility that it will have no effect, today’s climate models do not include permafrost melting effects. 

As a result, even if Watson were to incorporate such a model in its projections of weather over the next couple of years at business locations, it would be very likely to fail to anticipate the extremes of weather that would actually occur.  Thus, Heidi Cullen, another meteorologist savvy about climate change, wrote a book a few years back in which she projected that NYC would see a major hurricane with a dangerous rise in the water level that threatened the city – but with a near-miss in 2017 and with a greater Hurricane Sandy equivalent happening in 2041.

A Quick Fix

So what’s a poor business to do?  Well, as of now, someone within the organization needs to develop enough expertise in today’s climatology to be able to create an informal “model” based on current research that can project possible regional climate changes for business and sales locations over the next five years – assuming non-zero global-warming effects of “known unknowns.”  Then, the business needs to run “what-if” weather scenarios that include these new possible extreme weather events – high winds, new hurricane locations, extreme rain and snow, extreme drought – and also their ancillary effects – water shortages, stressed utilities, difficulties getting to stores and work, impoverished consumers.  TWC’s expertise would be critical in determining the “penumbra” of related weather – geographical size and track of an extreme storm, frequency/length of extreme droughts – while IBM’s analytics would scrutinize the unfolding scenario for business impacts of ancillary effects.  This approach would do a far better job of adequately preparing the firm for the medium term (say, 2-5 years). 

Ah, but then there’s the longer term, 5-35 years out.  What prepares the firm for medium-term adaptation may very well prove to be more costly than adaptation that gives up certain locations as lost causes – and far, far more costly than the firm taking its fair part in effective global mitigation.

Getting Specific

What follows is necessarily speculative and US-centric, but should give an example of both medium-term and long-term business thinking.  Today’s “new climate” as it plays out over the next 2 ½ years is characterized by a modified “el Nino/la Nina” cycle and a new Arctic impact on weather, especially in winter.  High temperatures along various parts of the Pacific have attenuated el Nino and delayed its onset, but over the next 2 years, and possibly for longer, we should be in an el Nino phase characterized by higher temperatures than normal, especially in the western US.  At the same time, extensive warming of the Arctic combined with a mild el Nino should lead to significantly higher Arctic temperatures in the summer (and winter), and may redirect the cold and “extreme snowfall” of this winter to the European side of the Atlantic.

However, at the same time, Greenland ice melting has been putting a fresh-water “cap” on the area where the Gulf Stream dives down to the deep ocean, slowing and weakening the “force of flow” of the Gulf Stream.  At the same time, the greater increase in winter temperatures in the Arctic due to global warming compared to the continental US has made it easier for Arctic cold to “punch through” the jet stream and park over the eastern US for weeks at a time.  The increased warmth of that Arctic air means increased incidence of “extreme snowfalls” either in the eastern US or (in the other phase of the “Arctic oscillation”) in northern Europe.

Combining the two effects above with ongoing global warming, and we might project winters similar to the last 2 in the eastern US, but with a shorter snow season and slightly warmer temperatures.  The southern and western US may see significantly higher summer temperatures, and we cannot anticipate that the drought in the California area will be overcome by enough rain to replenish reservoirs, especially with little or no snowpack in the Sierras.  Finally, we might anticipate that there may be another out-of-season Hurricane Sandy or 75-80-mph nor’easter in the Northeast, with higher-wind-speed hurricanes in the South.  Overall, the globe should see a faster increase in temperatures year-round, leading to record high temperatures over much of the globe.  Businesses should review not only their dependence on water but also the reliability and cost-effectiveness of their air conditioning.

Beyond that, in the next 5-35 years, another main actor is sea level rise.  It is plausible that on average, global sea level will rise almost a foot over the next 35 years, and the slowing of the Gulf Stream means that sea level rise along the East Coast may even double that.  Sooner than expected, then, cities such as Miami and perhaps even Boston are under threat, and increased wave levels during more-violent storms mean faster coastal erosion and destruction.  Businesses should rethink their investments in locations less than 25 feet above sea level, in areas of high drought (western and possibly southern US), and in areas requiring high levels of air conditioning costs.  Analytics such as that available via IBM/TWC plus “what-if” scenarios should allow fine-tuning these shifts in investments within regions.

The Bottom Line

One line credited, I believe, to an IBM employee way back when, said of programs in general what could well be said of “business as usual” weather-related analytics:  “Garbage in, garbage out.”  In other words, if you assume that future weather will be like the past, with today’s climate change, that assumption is untrue “garbage”; and the result of your analytics will be either a 5-day horizon that gives you that long to make major changes in your current setup at various locations, or a very misleading guide to your multi-year strategy’s chance of avoiding major unnecessary weather-related costs – a medium-term forecast that is “garbage” as far as effective planning is concerned.

I have suggested one approach to overcome this problem; but, really, the first step is to recognize that dealing “just in time” with a weather problem is far less than half the battle, and to do something about it.  If you do create an approach along the lines I have suggested, however, IBM/TWC is an exceptional partner, because TWC “gets” climate change and because IBM via Watson and the like can dig deeper for related business effects, once the data it is fed is no longer “garbage.”

One final thought:  I and others like Joe Romm have said that mitigation (avoidance of further carbon emissions and hence far greater climate change) was and is far less expensive than adaptation such as the steps we have talked about here.  Unfortunately, in the 5-7 years since this point was made, most businesses have failed to have much positive impact on mitigation, either in their investments or politically.  As a result, the costs of mitigation have not increased significantly, but the costs of the adaptation that has now become necessary have risen quite a bit.  To put it another way, ignoring mitigation has guaranteed much more in the way of adaptation costs – and we may anticipate the same happening over the next 5-35 years, unless most businesses change both their political and investment practices in a much more drastic way.