In Pictures: 18 essential Hadoop tools for crunching big data

Making the most of this powerful MapReduce platform means mastering a vibrant ecosystem of quickly evolving code

In Pictures: 18 essential Hadoop tools for crunching big data prev next

Loading...

Spark The future is already coming. For some algorithms, Hadoop can be slow because it generally relies on data stored on disk. That's acceptable when you're processing log files that are only read once, but all of that loading can be a slog when you're accessing data again and again, as is common in some artificial intelligence programs. Spark is the next generation. It works like Hadoop but with data that's cached in memory. The illustration at left, from Apache's documentation, shows just how much faster it can run in the right situations.

Spark is being incubated by Apache and is available from http://spark.incubator.apache.org/.

Prev More slideshows 19/19

Comments on this image

There are currently no comments for this image.

Comments are now closed.

Close

In Pictures: 18 essential Hadoop tools for crunching big data

19 images
Shopping.com

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?