When data centers were filled with racks of a kilowatt or two apiece, you could get away with inefficient cooling. Sure, the power bill may have been higher than it needed to be, but it wasn't a make-or-break issue.
Those naïve days, however, are long behind us. With 5 kW, 10 kW, 20 kW and even 30 kW racks widely available, there is no room for wasted cooling or poor air circulation. Yet, many data centers are doing just that.
Take the case of the Danfoss data center in Denmark. Built on the ground floor of a building several decades old, it has a couple of large concrete beams running in parallel along the ceiling. These beams, in effect, create several air pockets running the length of the room and trap about four feet of hot air.
"The top servers of our old racks were often overheating," said Jan Petersen, IT production manager at Danfoss A/S. "It was very difficult to get cold air to the top of the rack."
According to his figures, the top units were reaching temperatures of 25 to 29 degrees C at the front and were outputting air at 40 degrees C.
When the facility decided to implement HP blades, therefore, it installed an InfraStruXure system by APC-MGE of West Kingston, R.I. This cooling system provides complete hot-aisle containment, which means no hot air from two enclosed racks is allowed to spill out into the room.
Danish Farmer Makes Good
Mads Clausen, the son of a local farmer, founded Danfoss in the 1930s. He seized an opportunity afforded by the Great Depression. Due to restrictive customs practices, U.S. goods were shut out of the Danish market. Thus, Clausen began manufacturing valves and refrigeration gear.
From those humble beginnings, Danfoss has become a major force in worldwide industry. It now produces 200,000 components per day across 100 countries and employs a staff of 18,000 people. Its products can be broadly classified as being for the refrigeration, AC, heating and motor controls industries.
A couple of years ago, the data center included mainly racks of Compaq servers in the zero kW to three kW per rack class. Three computer room air conditioning units situated at the perimeter of the space cooled these machines. Although the units produced enough cold air to comfortably cool the Compaq racks via a raised-floor arrangement, concrete ceiling beams blocked hot-air recirculation, and pockets of warm air stubbornly refused to dissipate.
"As a result, we had some parts of the room with plenty of cold air, and other parts with very little," says Petersen. "We even had negative pressure in some cold aisles, i.e., the air was being sucked back down into the floor rather than fed to the front of the servers."
Not surprisingly, some servers were failing especially those in the upper reaches of the racks.
Move forward to more recent times. The data center added a SAN consisting of Brocade Silkworm switches and several disk arrays from EMC and HDS. It also added a massive StorageTek robotic tape library (model 9940 STK) with room for about 6,000 tape cartridges inside to back up about 760 TB daily from 34 locations throughout the world.
On the server side, Danfoss expanded to around 1,400 servers split between two facilities situated about a half-mile apart the main Danfoss data center and a smaller secondary site. In the past year, it has added mainly HP DL 380 rack servers and HP BladeSystem blades.
"We've exclusively purchased only HP servers for three years now and are very happy with them," said Petersen.
As this acquisition brought cooling demands through the roof, the IS organization realized it would have to radically shift its cooling framework to prevent severe overheating and keep power bills down. Thus, it installed several APC InRow RC units in an InfraStruXure hot-aisle containment cube. In this configuration, the hot aisle itself is enclosed and the hot air is fed back into the APC coolers, rather than left to drift back into the room.
"Our servers are no longer overheating," said Petersen. "When we add a rack of blades, we always add two InRow RC units one on each side."
A total of 23 racks of HP rack servers (approximately 4 kW to 6 kW per rack) have now been added to the hot-aisle containment cube, as well as 5 racks of HP blades (each 10 kW to 15 kW) and 11 APC cooling units. Most of these coolers are situated close to the blade racks. Two APC power racks have also been placed within the cube for power input and distribution, and Petersen said air from the coolers is blasted around the room to supplement the work of the computer room air conditioning units.
Chilled water pipes are brought under the floor to provide water to the InfraStruXure system. The temperature inside the cube is maintained at 27 degrees C. Outside air is 21 degrees C. Petersen said he used to run the computer room air conditioning units at 16 degrees C to keep some of the servers in the low 20s in temperature.
"Our server manager wasn't a fan of blades at first, but now he is," said Petersen. "About 60 percent of our current processing power comes from our two new aisles, yet they are still only about half full."
New Data Center?
Danfoss IT managers realize the existing spaces are inadequate. While they are actively planning a move to a new larger facility, they also understand that financial constraints may require them to remain in the current rooms a while longer. If necessary, the IS organizations is ready to replace the old Compaq racks with two more racks of HP rack and blade servers along with more APC RC units in a hot-aisle containment cube.
"We are probably moving to a new larger data center, but costs may keep us here a while longer," said Petersen. We plan to have dual everything at the new facility and continue to use HP servers as well as InfraStruXure cooling systems to keep cooling costs low."
In terms of UPS, the facility has an aging system located in the cellar. It has a capacity of 160 kW and is almost to its limit. Thus, Danfoss has limited ability to power many more servers, and the shift to a new data center building will have to happen soon.
"Two years ago, we received overheating alerts quite often," said Petersen. "Now, we have control over the environment. We could never have put blades into old environment due to their heat output."