Hadoop alternative to be open sourced

Information provider LexisNexis is releasing its HPCC Systems technology, but it may have an uphill battle

LexisNexis is planning to release its internally developed supercomputing platform as open source, providing developers with an alternative to the Hadoop framework for large-scale data processing, the company said Wednesday.

LexisNexis has been developing the technology, dubbed HPCC Systems, for the past 10 years, according to the company, which provides a variety of information services to legal firms, libraries, corporations and government entities.

"We've been doing this quietly for years for our customers with great success. We are now excited to present it to the community to spur greater adoption," said James Peck, CEO of LexisNexis' Risk Solutions division, in a statement. "We look forward to leveraging the innovation of the open source community to further the development of the platform for the benefit of our customers and the community."

HPCC Systems runs on clusters of commodity hardware and is made up of a number of components, centered around the company's Enterprise Control Language, a "declarative, data-centric programming language optimized for large-scale data management and query processing," LexisNexis said.

A component called Thor handles data ETL (extraction, transformation and loading) chores, while a third system named Roxie delivers "highly scalable, high-performance online query processing and data warehouse capabilities," LexisNexis said.

The system is able to analyze petabyte-sized volumes of data "significantly faster and more accurately than current technology systems," scaling up to thousands of nodes, the company said.

LexisNexis will offer both a community edition and commercial enterprise edition of HPCC Systems, which will be overseen by company CTO Armando Escalante.

At first, HPCC Systems will be offered as a virtual machine for testing by the community, with full binaries and the source code to be issued a number of weeks later.

The community edition will be released under the GNU Affero GPL v3 license. New code contributed by LexisNexis and community members will go to the open-source edition first, according to a detailed FAQ document on the company's site.

However, LexisNexis stressed that HPCC Systems won't involve the release of any of its "data sources, data products, the unique data linking technology, or any of the linking applications that are built into its products."

The community edition will also have a number of limitations compared to the enterprise edition, such as a restriction of one Thor process per node, according to a comparison chart. It will also get only "basic testing against different Linux distributions," while the enterprise edition will undergo a much more rigorous certification.

Pricing for the enterprise edition, which is offered with a number of support tiers, was not available.

Enterprise Edition subscription customers have access to a number of add-on modules as well, including a tool that converts the Pig Latin language used in Hadoop to ECL.

ECL specifications will be released under a Creative Commons license, the company said.

Despite HPCC Systems' pedigree and apparent long-term success as an operational system for LexisNexis, it will have an uphill battle in the broader marketplace, according to Forrester Research analyst James Kobielus.

"I don't doubt [HPCC Systems] does what they say it does, but they're late to the game. I don't know that they are going to get a lot of traction, because a lot of vendors and users have placed their bets with Hadoop," he said.

LexisNexis will also be hindered by the fact that it is not a database or data warehousing company, and despite its claims about HPCC Systems' capabilities, the company "is not going to win a performance battle against the whole Hadoop community," Kobielus said. "Venture capital money is going into startups left and right to involve that community."

Chris Kanaracus covers enterprise software and general technology breaking news for The IDG News Service. Chris's email address is Chris_Kanaracus@idg.com

Join the Good Gear Guide newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags business issuesapplicationshadoopLexisNexis Risk Solutionssoftwaredata warehousingbusiness intelligence

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Chris Kanaracus

IDG News Service
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Armand Abogado

HP OfficeJet 250 Mobile Printer

Wireless printing from my iPhone was also a handy feature, the whole experience was quick and seamless with no setup requirements - accessed through the default iOS printing menu options.

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?