This project is all the rage these days and is synonymous with “big data,” in which enterprises and Web properties sift through reams of data to surface insights about customers and users. Hadoop provides an operating system for distributed computing.
“If you want to run computations on hundreds of thousands of computers instead of just on one computer, Hadoop lets you do that,” says Doug Cutting, a primary contributor to Hadoop for several years. Hadoop arose from the Nutch Web software project in 2006, Cutting said. Companies like Cloudera, where Cutting is employed, and HortonWorks are building businesses around Hadoop. Future improvements will include boosts for security and scalability.