Purdue app slows servers when cooling fails

An administrator from Purdue University has developed software that can slow servers when the air conditioning goes out

While chip manufacturers continue to make their processors ever more powerful, at least one customer has found it useful to slow these chips down, at least long enough to keep them running when the data center air conditioning falters.

Patrick Finnegan, a systems administrator at Purdue University, has developed software that slows the clock speed of server processors, a throttling that reduces the heat they produce.

"Previously our only options were to put in a few large fans and hope that was enough, or start turning servers off," said Mike Shuey, who oversees Purdue's supercomputers. "This software gives us a middle ground that gets us by many outages."

Purdue is now reselling the software for US$250, through FolioDirect, an online e-commerce service for educational institutions.

With most commodity servers, once their ambient temperatures reaches a certain point, usually around 32 Celsius (About 90 degrees Fahrenheit), they will automatically shut off to prevent damage from overheating. Smart administrators will turn them off ahead of that, at least to facilitate a graceful shutdown.

In the world of academic supercomputing these restarts can be deadly, though. Purdue's clusters run many serial jobs that can take days, weeks, or even months to complete. And while some programs have frequent setpoints to which they can return that are close to where they at shutdown, many do not. One Purdue researcher, for instance runs atmospheric climate models that can require four months of continuous computing time.

"If our only recourse to survive an outage is to start turning off machines, we can throw away from two to three million[m] CPU hours of work," Shuey said. "It can take weeks and weeks of run time just to get back to the state we were in the minute before we turned things off."

In contrast, by throttling back the servers, the programs are slowed, but no work is lost.

Finnegan built the software using a clock frequency scaling driver available for the Linux kernel, which can control both Intel and AMD chipsets with frequency scaling capabilities. The software also relies on Altair job scheduling software as well as a set of cluster management tools from the U.S. Department of Energy's Oak Ridge National Laboratory.

As far as Shuey knows, no other software is available to do this task, either open source or commercial, at least for large clusters of servers.

Overall, the Purdue data center runs around 15,000 processors, mostly across two supercomputer clusters. One, called Coates, supplied by Hewlett-Packard, runs just under 8,000 processors from AMD. The other, a Dell-supplied configuration nicknamed Steele, runs 5,600 Intel processors.

The Purdue team estimates that power usage by processors can be cut by as much as 10 percent on Intel processors and by as much as 30 percent on AMD processors. The amount of power a server uses usually directly correlates to the amount of cooling needed.

"They may lose 70 to 80 percent performance, but we get a 30 percent power savings," Shuey said.

At least in its current incarnation, the data center's plan for cooling outages still requires a human in the loop.

The facility is cooled by chilled water piped in from the school's main cooling plant. The optimal temperature for the building is about 21 degrees Celsius (or about 70 degrees Fahrenheit). The data center uses an APC temperature monitoring system, which sets off alarms should the temperature go above 26 degrees Celsius (or about 80 degrees Fahrenheit). Should the alarm go off, the administrator can use the software console to throttle back the servers.

Since Finnegan wrote the software earlier this year, the school, located at West Lafayette, Indiana, has had to cut back server speeds twice, due to a combination of planned maintenance-related outages and a hotter-than-usual summer. Both times, the throttling worked as planned.

"The compute jobs slowed down, but the data center temperatures dropped," Shuey said. "It's much better to have jobs run slowly for an hour rather than throw away everyone's work in progress and mobilize staff to try to fix things," Shuey says.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Join the Good Gear Guide newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags ClusterssupercomputersserversNetworkingAMDhardware systemsintelenvironmentHewlett-PackardDellGreen data centerHigh performance

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Armand Abogado

HP OfficeJet 250 Mobile Printer

Wireless printing from my iPhone was also a handy feature, the whole experience was quick and seamless with no setup requirements - accessed through the default iOS printing menu options.

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?