IT & the Smart Grid Data Deluge

Bruce Rayner, Contributing Editor, Enterprise Efficiency OEM | 3/23/2012 | 16 comments

Bruce Rayner
When most people hear the term smart grid, they think of smart meters. But those little two-way digital devices that local utilities provide households are just one small, albeit important, component of the smart grid.

The smart grid encompasses the digitization and connectivity of everything having to do with the generation, delivery, management, and use of electricity.

“It’s applying network and IT infrastructure to the traditional operational side of delivering energy and provides for the collection and analysis of measurement data,” said Amy Price, a Dell senior consultant in the Storage Products Group.

Capturing this measurement data will allow utilities to improve the real-time control of the grid, analyze and respond to trends in energy usage, integrate alternate energy sources, and provide customers more buying and usage options. Among other things.

The grand vision is that smart grid technology will lead to greater efficiency, reliability, and sustainability of the utility infrastructure and conservation of natural resources. Some believe it will transform the power sector just as profoundly as the shift from analog to digital phones transformed telecommunications -- in ways we can’t even imagine today.

And not a moment too soon, as global energy demand is expected to increase by 75 percent or more within the next 20 years.

In addition to energy, smart grid projects are underway in the natural gas and water utility sectors that offer the potential for similar gains in efficiency and resource conservation.

The smart grid is a huge business opportunity for OEMs and IT companies looking to provide the enabling hardware, software, and services. From energy storage to data analytic to inverters, there’s a land grab underway to stake out a market position in every conceivable niche. Check out this diagram from Green Tech Media to see just some of the companies vying for a piece of the pie, which GTM estimates will reach nearly $10 billion this year in the US.

At the center of it all is -- you guessed it -- the challenge of how to manage and store all the data that will be flowing across the smart grid network. It's what Dell’s Price calls the “data deluge.”

And when she says deluge, she’s not kidding. Take Austin Power, the electric utility for the City of Austin, Texas, which is in Dell’s backyard.

According to Price, Austin Power processed about 20 terabytes of data a year in the dumb old days. Then in the mid-2000s, the utility installed smart meters citywide that collected and transmitted data every 15 minutes and stored it for three years. As a result, the annual data figure increased five times to 100 terabytes. Today, Austin Power is taking smart meter readings every 5 minutes and storing the data for seven years, which bumps that figure up to 400 terabytes a year. Add in real-time data and transmission data management requirements, and Austin Power is managing and storing 1.2 petabytes a year. That’s a 60X jump from the pre-smart grid era.

Now, multiply 1.2 petabytes by the 300-plus smart grid programs underway across the US, and you’re looking at close to 400 petabytes a year. Then extrapolate the data requirements to the global level and look down the road a few years, and we’re talking zettabytes. A zettabyte is one byte with 21 zeros.

What does this mean? For one thing, bigger budgets for IT departments at most major utilities. But more importantly, it means utilities like Austin Power have to elevate the role of IT, create a whole new set of capabilities, and develop expertise in areas that they never had, or needed, in the past. The CIO should be working shoulder to shoulder with the COO. More than a technologic shift, this represents a massive cultural shift in the way utilities need to operate.

But while painful and slow, cultural shifts can lead to remarkable transformations. Just think back to what life was like when telephones were bolted to the wall and had curly cords.

View Comments: Newest First | Oldest First | Threaded View
<<   <   Page 2 / 2
Bruce Rayner   IT & the Smart Grid Data Deluge   3/23/2012 7:55:41 PM
Re: hee-hee
@ Sara - are you kidding me?!?!?!? Of course Austin Powers is concerned about energy efficiency. He's from Canada, eh.
User Ranking: Blogger
Sara Peters   IT & the Smart Grid Data Deluge   3/23/2012 6:01:50 PM
Re: Get An Umbrella
Well Dave we won't know how that information will change behavior until we have the information in the first place, right? As Bill notes, utilities companies may soon need to employ a whole set of new IT staff who are specialists in crunching and making use of this data. I'd expect that some industries that use huge amounts of power -- like manufacturing, perhaps -- will follow suit.
Sara Peters   IT & the Smart Grid Data Deluge   3/23/2012 5:51:54 PM
What do you think Austin Power thought when Austin Powers came out?

Cause this guy doesn't really look serious about energy conservation

David Wagner   IT & the Smart Grid Data Deluge   3/23/2012 5:34:48 PM
Re: Get An Umbrella
Totally agreed, Curt. I just question the need for real-time data versus every 15 minute data or some of the other choices Bruce brings up. And I question precisely how much the knowledge itself changes the habits.

CurtisFranklin   IT & the Smart Grid Data Deluge   3/23/2012 5:30:30 PM
Re: Get An Umbrella
@David Wagner wrote:

...unless I have wildly variable power usage that I'm unaware of, I think people believe too much in the magic of the smart grid.

Dave, this is true if you live in a very temperate climate. But for those of us who live in places where there can be some real weather extremes, a smart grid can make a huge difference by allowing utilities to work with customers to do load leveling and intelligent overload protection. It really comes down to this: Which would we rather invest in, a smarter grid or lots more generating plants? I'd vote for the grid, especially since there are other benefits that can come from intelligence.
David Wagner   IT & the Smart Grid Data Deluge   3/23/2012 3:04:12 PM
Get An Umbrella
Bruce, I think you're right about the deluge coming. But i question the real value of the smart grid. Sure, there is some obvious low hanging fruit like discovering more precisely the increased power needs when the temperature rises into the 80's or something.

But I have to say, unless I have wildly variable power usage that I'm unaware of, I think people believe too much in the magic of the smart grid. This idea that suddenly knowing your power using down to the tiniest watt is going to instantly sav eus money seems specious.

I know how many miles to the gallon my car gets, but it doesn't change where i drive.
<<   <   Page 2 / 2

The blogs and comments posted on do not reflect the views of TechWeb,, or its sponsors., TechWeb, and its sponsors do not assume responsibility for any comments, claims, or opinions made by authors and bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.

More Blogs from Bruce Rayner
Bruce Rayner   7/20/2012   21 comments
Earlier this month we witnessed a little scuffle involving Apple that sheds light on perhaps the most important challenge the electronics industry faces: how to balance environmental ...
Bruce Rayner   7/13/2012   9 comments
The tectonic plates of global manufacturing are starting to shift -- and the consequences could be dramatic.
Bruce Rayner   6/29/2012   74 comments
There was a piece in today's New York Times about Google's domestic manufacturing experiment. Seems the company's Nexus Q, a just-released home media player, is not only being designed in ...
Bruce Rayner   6/15/2012   15 comments
When we think of global companies, we typically think of the Fortune 500. These are big companies with their names plastered on shiny office buildings in cities like Berlin, London, ...
Bruce Rayner   6/8/2012   3 comments
Last year, 2011, was a calamitous year for OEMs as weird weather wreaked havoc on their supply chains. Perhaps the worst climate-related catastrophe from a supply-chain standpoint was the ...
E2 IT Migration Zones
IT Migration Zone - UK
Why PowerShell Is Important
Reduce the Windows 8 Footprint for VDI
Rethinking Storage Management
IT Migration Zone - FR
SQL Server : 240 To de mémoire flash pour votre data warehouse
Quand Office vient booster les revenus Cloud et Android de Microsoft
Windows Phone : Nokia veut davantage d'applications (et les utilisateurs aussi)
IT Migration Zone - DE
Cloud Computing: Warum Unternehmen trotz NSA auf die „private“ Wolke setzen sollten
Cloud Computing bleibt Wachstumsmarkt – Windows Azure ist Vorreiter
Like Us on Facebook
Twitter Feed
Enterprise Efficiency Twitter Feed
Site Moderators Wanted
Enterprise Efficiency is looking for engaged readers to moderate the message boards on this site. Engage in high-IQ conversations with IT industry leaders; earn kudos and perks. Interested? E-mail:
A Video Case Study – Translational Genomics Research Institute
e2 OEM Video

On the Case
TGen IT: Where We're Going Next

7|11|12   |   08:12   |   10 comments

Now that TGen has broken new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions, the company discusses what will come next for it and for personalized medicine.
On the Case
Better Care Through Better Communications

6|6|12   |   02:24   |   11 comments

The achievements of the TGen/Dell project could improve how all people receive healthcare, because they are creating ways to improve end-to-end communication of medical data.
On the Case
TGen IT: Where We Are Now

5|15|12   |   06:58   |   6 comments

TGen is breaking new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions.
On the Case
TGen IT: Where We Were

4|27|12   |   06:45   |   10 comments

The Translational Genomics Research Institute wanted to save lives, but its efforts were hobbled by immense computing challenges related to collecting, processing, sharing, and storing enormous amounts of data.
On the Case
1,200% Faster

4|18|12   |   02:27   |   12 comments

Through their partnership, Dell and TGen have increased the speed of TGen’s medical research by 1,200 percent.
On the Case
IT May Improve Children's Chances of Survival

4|17|12   |   02:12   |   8 comments

IT is helping medical researchers reach breakthroughs in a way and pace never seen before.
On the Case
Medical Advances in the Cloud

4|10|12   |   1:25   |   5 comments

TGen and Dell are pushing the boundaries of computing, and harnessing the power of the cloud to improve healthcare.
On the Case
TGen: Living the Mission

4|9|12   |   2:25   |   3 comments

TGen's CIO puts the organizational mission at the heart of everything the IT staff does.
On the Case
TGen Speeding Up Biomedical Research to Save More Lives

4|5|12   |   1:59   |   6 comments

The Translational Genomics Research Institute is revamping its computing to improve speed, storage, and collaboration – and, most importantly, to save lives.
On the Case
Computing Power Helping to Save Children's Lives

3|28|12   |   2:13   |   3 comments

The Translational Genomics Institute’s partnership with Dell is enabling them to treat kids with neuroblastoma more quickly and save more lives.
Curtis Franklin Jr.
OEMs Change Roles

1|18|13   |   1:55   |   3 comments

OEMs can change markets – here's why IT should have a say in the decision.
Tom Nolle
The Enterprise Side of Amazon Fire

9|29|11   |   2:04   |   6 comments

Amazon Fire’s split-browser model hosts some of the GUI in the cloud, which could have a major impact on virtual desktop thinking.
Curtis Franklin Jr.
The OEM Relationship

9|13|11   |   02:02   |   1 comment

The growth of OEM relationships means that enterprise IT execs must pay closer attention to who's responsible for support and development.
Pablo Valerio
Can't Land on the Runway Behind You

8|15|11   |   1:36   |   1 comment

One lesson from aviation also applies to big IT projects: Give yourself plenty of leeway and have room to maneuver.
Ivan Schneider
Flecksequence Explained

7|28|11   |   2:46   |   3 comments

How to use the term in a sentence and, more importantly, how flecksequence can help manufacturers.
Sara Peters
E2 Has a New Look!

7|20|11   |   2:53   |   6 comments

E2's gotten a makeover. Take a tour through some of our new features.