SC2012: Top500 expects exascale computing by 2020

But supercomputing software will have to be designed differently to handle exascale workloads, researchers at SC2012 warn

If the increase in supercomputer speeds continue at their current pace, we will see the first exascale machine by 2020, estimated the maintainers of the Top500 compilation of the world's fastest systems.

System architects of such large computers, however, will face a number of critical issues, a keeper of the list warns.

"The challenges will be substantial for delivering the machine," said Jack Dongarra, a University of Tennessee, Knoxville, researcher who is one of the principals behind the Top500. Dongarra spoke at the SC2012 conference, being held this week in Salt Lake City, during a presentation about the latest edition of the list, released Monday.

We still have a way to go before exascale performance is possible. An exascale machine would be capable of one quintillion FLOPS (floating point operations per second), or 10 to the 18th FLOPS. Even today's fastest supercomputers offer less than 20 percent of the capability of an exascale machine.

In the most recent edition of the Top500 list of supercomputers, released Monday, the fastest computer on the list was the Oak Ridge National Laboratory Titan system, a machine capable of executing 17.59 petaflops. A petaflop is a quadrillion floating point calculations per second, or 10 to the 15th FLOPS.

But each new Top500 -- the list that is compiled twice a year -- shows how quickly the speeds of supercomputers grow. Judging from the list, supercomputers seem to gain tenfold in power every 10 years or so. In 1996, the first teraflop computer appeared on the Top500, and in 2008, the first petaflop computer appeared on the list. Extrapolating from this rate of progress, Dongarra estimates that exascale computing should arrive around 2020.

The High Performance Computing (HPC) community has taken on exascale computing as a major milestone. Intel has created a line of massively multicore processors, called Phi, that the company hopes could serve as the basis of exascale computers that could be running by 2018.

In his talk, Dongarra sketched out the characteristics of an exascale machine. Such a machine will likely have somewhere between 100,000 and 1,000,000 nodes and will be able to execute up to a billion threads at any given time. Individual node performance should be between 1.5 and 15 teraflops and interconnects will need to have throughputs of 200 to 400 gigabytes per second.

Supercomputer makers will have to construct their machines so that their cost and power consumption do not increase in a linear fashion along with performance, lest they grow too expensive to purchase and run, Dongarra said. An exascale machine should cost about $200 million, and use only about 20 megawatts, or about 50 gigaflops per watt.

Dongarra expects that half the cost of building such a computer would be earmarked for buying memory for the system. Judging from the roadmaps of memory manufacturers, Dongarra estimated that $100 million would purchase between 32 petabytes to 64 petabytes of memory by 2020.

In addition to challenges in hardware, designers of exascale supercomputers must also grapple with software issues. One issue will be synchronization, Dongarra said. Today's machines pass tasks among many different nodes, though this approach needs to be streamlined as the number of nodes increases.

"Today, our model for parallel processing is a fork/join model, but you can't do that at [the exascale] level of a parallelism. We have to change our model. We have to be more synchronous," Dongarra said. Along the same lines, algorithms need to be developed that reduce the amount of overall communication among nodes.

Other factors must be considered as well. The software must come with built-in routines for optimization. "We can't rely on the user setting the right knobs and dials to get the software to run anywhere near peak performance," Dongarra said. Fault resilience will be another important feature, as will reproducibility of results, or the guarantee that a complex calculation will produce the exact same answer when run more than once.

Reproducibility may seem like an obvious trait for a computer. But in fact, it can be a challenge for huge calculations on multinode supercomputers.

"From the standpoint of numerical methods, it is hard to guarantee bit-wise reproducibility," Dongarra said. "The primary problem is in doing a reduction -- a summing up of numbers in parallel. If I can't guarantee the order in which those numbers come together, I'll have different round-off errors. That small difference can be magnified in a way that can cause answers to diverge catastrophically," he said.

"We have to come up with a scenario in which we can guarantee the order in which those operations are done, so we can guarantee we have the same results," Dongarra said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags ClusterssupercomputersHigh performanceIBMhardware systemsCray

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Most Popular Reviews

Latest News Articles


PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?