It’s Raining Data
Connected Futures
Executive Insights by Cisco
by Kevin Delaney
Big data gives companies a better handle on how day-to-day weather affects the bottom line.
Mega-storms like Harvey, Irma, and Maria remind us of the awesome power of nature.
But while we can’t change the weather, we can be proactive about it — in ways that weren’t possible even a few years ago.
Smart businesses — in industries from agriculture and insurance to transportation and retail — are using digital tools to save lives, protect property, and capture competitive advantage.
“Without doubt, data analytics and environmental data have huge potential across many markets,” said Brad Colman, director of weather science for the Climate Corporation.
That means not only better predictions for massive storms, but also a deeper understanding of how day-to-day weather affects workers, shoppers, farmers, and equipment.
Climate Corporation, for example, offers pinpoint insights for the agricultural industry by combining terabytes of weather data with other streams — including from Internet of Things sensors arrayed across farms and in soil. The result is a detailed field-level picture, influencing decisions like how much nitrogen or pesticide to apply to a specific area of a farm, or when to prevent a plant disease from ruining a crop.
That kind of insight goes far beyond the already impressive predictions available to anyone with a TV or weather app.
“The average person’s requirement isn’t all that great,” said Lee McArthur of 5Fathom, which provides weather analytics for the insurance industry. “Just give me an idea if it’s going to rain or not today. Do I need to bring a rain jacket?
“But in the enterprise space, it’s all about the understanding of what you need for your business.”
Precise predictions — tailored to industry needs — can mean the difference between profit and loss.
Jim Menard, head of travel & transport, and energy and utilities for The Weather Company, stressed how weather affects the delicate choreography of a shipping hub.
“Getting a ship from point A to point B,” Menard said, “becomes an issue of not only what’s the shortest distance, but also which way are the winds most favorable? Which way are the seas most favorable? Which way are the ocean currents most favorable?”
A late container ship can spell cascading trouble in what Menard calls the “ultimate logistics hubs,” ports where air, rail, trucking, and shipping all meet.
“Running a port can become a logistics nightmare, and weather is often the reason why things go awry,” Menard added. “Giving these people as much notification if something’s going to happen and then a prescription for what to do is pretty important.”
A Hail Storm Is Coming — to Your Street
With more and more data points capturing weather information — including radars, satellites, ocean buoys, and even backyard weather stations and smart phones — coupled with powerful networks and analytics, predictions become hyper-local.
“We routinely run, in some cases, models down to under one kilometer in spacing,” Menard said. “You’re down to almost street level in being able to forecast: Where is the cloud bank? Where is the fog bank going to be? Where is that squall line likely to hit?”
That can be especially valuable to an insurer or power utility, especially when historic weather-impact data is combined with real-time streams.
“You now have artificial intelligence and machine learning to do predictive analytics and cognitive projections,” Menard said. “Then come up with, okay, you’re going to have 550 homes without electricity in this particular county in northwest Connecticut.”
5Fathom helps (re)insurers understand the impact of natural catastrophes and inclement weather through analytics. McArthur paints a scenario in which an insurer sends real-time prompts as a weather event targets a single neighborhood.
“If you are a policy holder with my insurance company,” McArthur added, “and I know that in the next 30 minutes there’s a very good chance of a hail storm coming to your residence, I’ll send you a text message and say, ‘Hey, you have about five minutes to move your car into the garage.’ Or, ‘We know you don’t have a garage or hail cover on your policy. So, respond yes for a quote, and respond 1 to add hail coverage to your policy right now.’ ”
Easy access to compute power and data storage make such responses possible.
Ever-decreasing storage costs “enable people to house more data. And then cloud-based computing helps to automate a lot of the analytics,” said David Frieberg of the retail-insights firm Planalytics.
Planalytics mines vast troves of historic and current weather data, then combines them with data on sales figures, floor traffic, and customer demographics and behavioral patterns. The result is a detailed picture of how weather affects a retailer’s performance. That can be critical for planning inventory, staffing, supply-chain logistics, and so on.
“Weather has a huge impact on consumer behavior,” said Frieberg, “How we buy, what we wear, what we do.”
But it doesn’t take a hurricane or blizzard to impact buying patterns.
“When we measure how much the weather moves sales up or down for a company,” said Evan Gold of Planalytics, “90 percent or more comes from things that aren’t extreme weather events. It’s just a bit cooler, it’s just a bit warmer, it’s been a rainier period versus a drier period. Those drive most of the weather-driven sales volatility.”
From Data Downpours, a Steady Drizzle of Insight
Parsing insights from multiple large data streams is no small challenge.“Compared to the past, we don’t have a data problem in terms of the availability,” said McArthur, “but the problem becomes, how do you do something with all of that data, at scale and at a resolution and cost that is meaningful to the business.”The power of the network is critical.
“In terms of chewing through petabytes of data,” said McArthur, “analytics technology requires a lot of very chatty network connectivity to be able to build models, or to complete a picture.”
Added Menard: “When you put out a tornado warning, there can’t be any delay. You have to have a network that can handle the load, but more important the immediacy and the reliability becomes paramount.”
Data quality is another concern whenever large streams are involved. In short, all data points aren’t created equal.
“You’re getting this data from all sorts of different sources,” said Menard. “Satellites, radars, in some cases personal weather sensors, lightning detectors. All this data comes together, but it’s all prone to some noise. You have to put all this data to some pretty intense quality control.”
Radar, for example, can mistake birds, floating debris, or wind turbines for storms. And a home weather station may be positioned improperly, throwing off temperature or wind readings.
Automation speeds the search for data anomalies.
“Through machine learning algorithms,” said Colman, “we can actually monitor where the quality is, and where we’re seeing bigger uncertainty where we look at matches with numerical weather prediction models.”
Telling Stories With Data
Making weather data work for a farmer, airline dispatcher, or retailer goes beyond algorithms and network capabilities. Data must be accessible, intuitive, and visual.“To tell a story with the data is more of an art form than a science,” McArthur said. “Because people need to be told the story in a way that’s contextual to their own business.”
That’s especially true when fast, highly informed decisions must be made.“You’ve got to distill it to some type of visualization or alert that is quick and to the point,” said Menard. “An air dispatcher doesn’t enjoy looking at a complicated upper-air chart. What they really want to know is ‘show me where it’s bad to fly right now.’ That’s where design becomes a huge factor.”
That’s as true in the field as it is in the air.
“How do we bring it together in a way that’s consumable and actionable?” Colman said. “We’re focused on presenting it in a way that actually can inform a farmer to make a decision that will save him money, make it more efficient, and improve his bottom line.”
Whatever the industry, weather promises to play an even greater role in coming years.
“We don’t have a lot more arable land,” said Colman. “And that’s really what the digital agricultural revolution is about, allowing farmers to do the most that they can. That requires high-quality environmental data, weather information.”
Meanwhile, what were once dubbed “storms of the century” show up with disturbing frequency. “The reality is, we are just in an environment with more volatile weather,” said Frieberg, “and businesses have to start to pay attention to it.”