Inside the Cloud: Data Centers and Energy Use

  • Highmark's data center near Harrisburg, PA houses 400 servers that process a million and a half online transactions a day. Photo: Lauren Knapp

  • At Highmark's data center, the hot air is circulated in the server room to reduce heat and reduce need for AC. Photo: Lauren Knapp

  • Highmark's power supply from the grid is backed up by banks of batteries that are always charging and a huge diesel-powered generator. Photo: Lauren Knapp

  • MG Berhe, Highmark's data center IT director, says the center has adapted some individual servers to work like multiple servers to save energy. Photo: Lauren Knapp

February 23, 2013

Every day millions of us pull out our laptops, smart phones and tablets and access trillions of tiny bits of electronic information that are stored in data centers around the country. Retrieving those random tidbits of data takes energy.

Dave Andersen, a professor in the computer science department at Carnegie Mellon University, offers an example of how online giants such as Facebook must look for random bits of data – “needles in a virtual haystack” – and pull them together on a Facebook page.

Andersen sits at his office computer and runs his finger down the screen as his Facebook profile appears.

“There are probably 30 pictures just on this little page: little tiny snippets of random information,” Andersen points out.

It turns out this random access from large sets of data is one of the most difficult things to do efficiently on a modern computer.

For the past few years, Andersen and some of his graduate students have been working to make retrieval of these snippets of information more efficient. To do that, they’ve created a way to slow down servers – the bulked up computers used at data centers – and speed up disks where data are stored. That way servers no longer have to waste energy waiting on slow disks.

Andersen figures their system is two or three times more energy-efficient at grabbing small bits of data than a conventional server.

Andersen, who is an avid rock climber and runner, says he’s always had a spot in his heart for doing research related to energy efficiency and computing. But he says a viable application for that kind of work didn’t present itself until the advent of data centers.

“Suddenly with data centers, we have so much computation packed into a single place but also that is usually owned and operated by a single entity who is footing the bill for the power and has an incentive to do something about it,” Andersen says.

Data centers are ripe for energy savings

Computer systems in data centers process and store a litany of electronic information: emails, online bills, texts, movies. These facilities – some as large as football fields and some as small as closets – are popping up everywhere. There are 20 to 30 thousand large centers in the United States alone. There are also several million smaller facilities.

Collectively, data centers use close to two percent of all the electricity we consume. EPA estimates that’s enough electricity to produce 60 metric tons of greenhouse gases. That is equal to the carbon dioxide emissions that a dozen or so coal-fired power plants produce a year.

Critics say a lot of that energy is wasted not only in inefficient computing software and hardware like Dave Andersen is addressing, but in the way the centers are operated and protected. Data center owners who pay millions of dollars a year in energy bills are starting to pay attention to saving energy.

A data center tour

Highmark is one of the biggest health insurance companies in the country. Company officials say they thought about saving energy and reducing their carbon footprint when they built their high security data center near Harrisburg, Pennsylvania in 2005. The LEED-certified center processes a million and a half transactions a day.

Tammy Petrasic, the facilities management director, and MG Berhe, the center’s head of information technology, both worked on the building’s design.

The exterior of the building is nondescript. Dozens of cameras watch the comings and goings throughout the facility.

Berhe and Petrasic pass through several rooms and a corridor to the data center floor. Berhe uses his security badge and fingerprint to gain access. The door opens and the loud roar of servers and air conditioning filters into the outer room. 

Inside, rows and rows of metal cabinets hold 400 servers on a raised floor about the size of an NBA basketball court. There are also data storage machines here.

The servers kick out a lot of heat. A 2007 EPA report to Congress estimated that half of the energy used in data centers is used to cool servers and data storage equipment.

The Highmark center was designed to circulate the hot air away from the servers.

“Because we circulate the air, that allows an advantage, so we don’t have to use the air conditioning system,” says Berhe. “We have to use it less with the circulation.”

Just a decade ago, server rooms were kept cooler. Now this room's temperature is maintained at 77 degrees. The servers are cooled with a mechanical air conditioner and a passive air exchange system that allows outdoor air in cooler seasons to chill the water in the AC.

The newer generation of servers used at this center is less temperature sensitive and more energy efficient. In fact, some individual servers here have been adapted to work like multiple servers.

“We’re operating more efficiently than 80 percent of the data centers across the country,” says Petrasic.

But Highmark’s electric bill is still around $800 thousand a year because the center has backup power supplies, redundant Internet connections and cooling equipment, and high-security systems. The company says the medical information they process round-the-clock is just too important to give up these safeguards.

“At this data center we’re going to continue to do what we can to save energy but you get to the point where you know there’s a point of no return where you have to maintain operations first,” Petrasic says.

A growing sector

Some energy experts see these layers of redundant equipment and power sources as overkill. But Pierre Delforge, who focuses on high-tech energy consumption for the Natural Resources Defense Council, is encouraged by energy advances in the data center industry. Nevertheless, he says there are many more savings to be had in big centers. He also says smaller facilities – spaces with fewer than 100 servers – need to do more to save energy.

“Collectively they are responsible for approximately half of all data center energy use in the US," says Delforge. 

Delforge says small centers can collectively save about $2 billion a year in energy bills by running fewer servers at full capacity and better managing air flow. Energy savings are important because his 2012 study for NRDC finds the data center sector is growing.

The report estimates the sector is growing between five to ten percent. It says, if left unchecked it could more than double by the end of the decade and become one of the largest users of electricity in the United States.

The federal government is beginning to consolidate some of its thousands of data centers and has started to work with the industry to measure and reduce energy use. Highmark, for example, reports to EPA once a month about electricity consumption. The hope is all these efforts will lower US data center bills and reduce the industry’s carbon footprint.