Lustre settles into post-Oracle life

A California startup picks up where Oracle leaves off in support for the Lustre file system

Despite reassurances from Oracle, advocates of yet another ex-Sun Microsystems technology are voicing concern about the future of their software. In this latest case, the technology is Lustre, a file system widely used across the supercomputing community.

"Lustre is in a bit of a flux at the moment. The community feels a little bit that Oracle is turning its back to them, and there is discussion going on over whether or not Oracle is forking the code," said Brent Gorda, CEO of Whamcloud, a San Francisco-based, venture capital-funded company recently started to service the potential market of HPC (high-performance computing), Linux-based Lustre users.

While Oracle has pledged to continue to develop the software, it has no plans to sell commercially supported versions of the software beyond version 1.8.3, the last released by Sun, except as part of integrated packages of Oracle hardware and software.

With Oracle potentially scaling back its plans to offer Lustre as widely as possible, the file system joins a number of other Sun technologies whose futures appear altered or cloudy since Oracle acquired Sun in January, such as OpenOffice (now renamed LibreOffice), OpenSolaris, OpenSSO and others.

The last public announcement the company made about its plans for the technology was at the Lustre User Group 2010 meeting in Monterey Bay, California, last April, according to Gorda. He said that Oracle has not made any announcements about Lustre since the meeting. "In the community, we're left wondering and assuming bad things," he said. Oracle declined to be interviewed for this article.

At that meeting, Oracle engineer Peter Bojanic gave the audience an overview of Oracle's plans for the technology.

In short, he explained that Oracle will continue to sell a commercially supported version of the current edition of the software, version 1.8, but has no plans to sell a standalone, software-only version of the next release, version 2.0. It will, however, continue to shepherd the development of the software and use it as part of integrated hardware/software offerings. Oracle released version 2.0 in August.

Much like IBM offers its own parallel file system, GPFS (General Parallel File System), for its high-performance computing systems, Oracle will maintain Lustre primarily for its own systems, Gorda said. Whamcloud sees its niche in supporting Linux-based Lustre deployments that run on commodity x86 servers.

"Because of the perception that Oracle is backing away from HPC and open Linux, there is a hole there, and we started Whamcloud to service that area," Gorda said.

Lustre is what is widely known as a massively parallel file system, meaning that it can be used for storing vast arrays of data across multiple nodes, by using a central index of where all the data is kept. The technology is used across a large percentage of the world's fastest supercomputers.

Beyond supercomputing, Gorda sees Lustre as a good fit for the emerging field of cloud computing. "Lustre has already proved itself to run at speeds that are incredibly demanding. I think of it as a superset of what cloud computing and data analytics needs," he said.

Lustre has multiple controllers, which means that there is less chance of having a bottleneck when lots of sources are seeking or writing data to disks. It has been shown to support more than 50,000 clients simultaneously. Lustre can deliver over 2GB/s (gigabits per second) to an individual client. It is an open-source technology, though in 2007 Sun Microsystems acquired CFS, the company that did much of the early development work.

"Lustre is a good, scalable, robust file system," said Mark Seager, principal investigator for supercomputing platforms at the U.S. Energy Department's Lawrence Livermore National Laboratory (LLNL). About 25 of the lab's systems, both for classified and unclassified work, run on the file system.

LLNL was the first large organization to use the file system in a production development, and continues to fund its development. LLNL also turned out to be one of Whamcloud's first customers. Last week, LLNL commissioned Whamcloud to undertake a number of Lustre-related tasks. One is upgrading the LLNL's LMT (Lustre Monitoring Tool) so that it can work with Lustre 2.0. A debugging tool, LMT allows an administrator to determine if one particular application is taking up too many resources.

"In a multi-user environment, when a file system gets too busy, it is hard to correlate what application is causing that," Gorda said. LLNL will make this software open source, and it is scheduled to be released early in 2011.

Whamcloud will also help LLNL characterize Lustre performance on solid-state disks, which are expected to be widely used in HPC systems as their storage needs continue to balloon. "We expect that we'll find performance issues inside Lustre that would be addressed in the long term," Gorda said. "We're getting a metric for determining if Flash [drives] are the right thing to use."

In contrast to Gorda's claims, LLNL's Seager has not felt that Lustre support has lagged since Oracle assumed control of the technology. Oracle has helped LLNL in matters of hardware maintenance and improving the way the failover mechanisms work. "We're very happy with the collaboration with Oracle," he said.

Still, Whamcloud plans to ramp up support for the technology.

"There's been this confusion for the past six months, and I think we're seeing a calm return to the community. People are seeing that Lustre is not going away," Gorda said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags Clusterssupercomputersstoragehardware systemssoftwarecloud computinginternetInfrastructure servicesOracleStorage ManagementHigh performance

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Most Popular Reviews

Latest News Articles

Resources

PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?