Sandia tests supercomputer virtualization

High-performance computer virtualization could provide more bang for the buck, researchers contend

The U.S. Energy Department's Sandia National Laboratories is investigating the possibility of using virtualization to allow its researchers to make better use of its behemoth Red Storm supercomputer. Researchers from Northwestern University and the University of New Mexico are also participating in the project.

"Our focus is to create a more flexible environment for supercomputer users by leveraging virtual machines," said researcher Kevin Pedretti, who heads up the project for Sandia. "The end-goal is to create a more flexible supercomputer environment for users, without sacrificing performance or scalability."

Pedretti said that, to his knowledge, this is the first systematic test of virtualization for HPC (high-performance computing) platforms. Sandia and the U.S. National Science Foundation funded the work.

Thus far, supercomputing hasn't made great use of virtualization. "The conventional wisdom in the HPC community is that virtualization has too high of an overhead to be useful," Pedretti said, in an e-mail interview. "We hoped to challenge this by showing reasonable overhead was possible even for tightly-coupled, communication-intense scientific applications running at a large scale in a virtual machine."

Virtualization could prove valuable, Pedretti contends, because researchers wishing to do large-scale simulations or number-crunches would no longer be bound to using only those applications that run on supercomputer operating systems.

In the team's tests, virtualized programs ran at 95 percent of the speed on bare metal.

To conduct the tests, the researchers got 12 hours of dedicated system time on the Sandia Red Storm supercomputer to conduct the tests. The tests were run on 6,240 quad-core compute nodes

They ran the applications in two modes -- real and virtual. To set a baseline, one run was done in the native Red Storm environment, which employs the Catamount Lightweight Kernel (LWK), a lightweight operating system.

For the virtual deployment, the researchers booted onto Red Storm nodes their own next generation version of LWK, called Kitten. To run a virtual machine, Kitten was embedded with the Palacios virtual machine monitor, developed by Northwestern University in Evanston, Illinois, and the University of New Mexico.

The virtual machines then booted the Catamount as the guest OS. The test applications were launched from Red Storm's application launch tool, as per the normal protocol. "From the user's perspective on the service node, there is no visible change to the normal Red Storm software environment," Pedretti said.

A range of different programs was tested, including simple benchmarking operations and two actual operational applications, Sandia's own CTH shock-wave simulation code and the SAIC Adaptive Grid Eulerian (SAGE) hydrodynamics simulation application.

The programs were spread across multiple nodes, from two nodes to all 6,240, depending on the program itself.

The researchers decided to test only virtualization software developed for the HPC computing environments because mainstream virtualization software, such as Xen, is oriented more toward commodity server consolidation rather than high-performance computing, Pedretti said. Also, using HPC software allows the researchers to fine tune the settings to such a degree that would not be possible with the mainstream hypervisors.

Red Storm was ranked as the 17th most powerful supercomputer in the world by the most recent compilation of the Top500.org listing of supercomputers. A Cray XT-based machine running Linux, Red Storm runs 12,960 Advanced Micro Devices Opteron processors (6,720 dual-core and 6,240 quad core for a total of 38,400 cores). The system has been demonstrated to calculate 204 TFlops (trillion floating-point operations per second).

This project is not Sandia's first foray into supercomputer virtualization. Last summer, two Sandia researchers investigated the use of the Lguest Linux hypervisor to launch 1 million nodes on another Sandia supercomputer, in order to replicate a massive botnet, for research purposes.

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags Sandia National LaboratoriesLguestsupercomputingvirtualisation

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Essentials

Microsoft L5V-00027 Sculpt Ergonomic Keyboard Desktop

Learn more >

Lexar® JumpDrive® S57 USB 3.0 flash drive

Learn more >

Mobile

Lexar® JumpDrive® S45 USB 3.0 flash drive 

Learn more >

Exec

Lexar® Professional 1800x microSDHC™/microSDXC™ UHS-II cards 

Learn more >

HD Pan/Tilt Wi-Fi Camera with Night Vision NC450

Learn more >

Audio-Technica ATH-ANC70 Noise Cancelling Headphones

Learn more >

Lexar® JumpDrive® C20c USB Type-C flash drive 

Learn more >

Budget

Back To Business Guide

Click for more ›

Most Popular Reviews

Latest News Articles

Resources

PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?