NPR deploys Splunk for Web analytics

Using Splunk, NPR has found a faster way to build user data statistics

In a novel use of the software, National Public Radio is using the Splunk log search engine to analyze Web traffic for its audio streams and downloads.

NPR metrics analyst Sondra Russell described the setup at GigaOm's Big Data conference, held Wednesday in New York.

Splunk offers what it calls a search engine for machine data. It was originally built to parse log files, or the files programs and hardware generate to document their transactions, errors and other operational information. By coordinating the timestamps of messages from different applications and hardware, Splunk allows system administrators to pinpoint difficult-to-locate system problems.

In recent years, however, customers have been expanding their uses of Splunk to other duties, explained Splunk Chief Technology Officer Erik Swan, also speaking at the event. Web traffic analysis and business intelligence are two such ancillary uses.

For much of its Web traffic monitoring, NPR uses standard Web traffic analytic software, which can deliver reports on how many people visit each Web page. Such software usually generates these counts by using cookies or by embedding each page with a small script that alerts the software when the page is rendered in a browser.

The media organization, however, found it difficult to get reliable usage summaries for a number of aspects of its service. For instance, the organization needed to get an accurate count of how many listeners tuned into their streamed audio and video programs.

To get this data, NPR had prepared a PHP script that would parse the server log files and translate the results into a form that could be digested by Adobe's Omniture, a Web analytic tool. Getting information back, however, could take up to 24 hours, and Russell still didn't trust that the results provided an accurate count.

In the cases of streaming usage, many users might start a stream, then pause it, and restart it. Or perhaps a user would restart a stream after a failed Internet connection. In the server log files, all these events were logged as separate events, not a linear sequence of actions by a single user. As a result, there was no way of determining how many connections were from different listeners, and how many were multiple streams to a single user.

"With our PHP scripts, we could not get that level of sophistication. So our numbers could be off," Russell said."It was hard to make rational decisions based on this."

By working with Splunk, NPR could derive listener numbers and information directly from its servers' log files. The software allows users to script search results and then graph the results, or show them on a dashboard.

In NPR's case, for instance, it provided the exact number of listeners for each program that was streamed or downloaded. "With Splunk you can wrap multiple [log entries] into a single distinct visit, so I feel my numbers are actually accurate," she said.

Splunk helped identify users' mobile platforms as well. An increasing amount of traffic to the NPR site comes from mobile clients, such as iPhones, iPads and Android smartphones. In one case, a manager wanted to know which version of the iPhone operating system was most often used, as the results would direct the company's design work for its iPhone app.

"I could not have answered that without Splunk," Russell said.

Splunk also solved a seemingly unsolvable problem for the organization: determining how to pay royalties for streamed songs. NPR offers a streaming service for songs, called SoundExchange. It must pay out royalties for each song played, based on the number of listeners that stream had at the moment. "It can get complicated to reconcile the two," Russell said.

Using Splunk, Russell was able to merge two files -- a list of when each song was played, and the number of listeners that stream had when the song was played. "Splunk put them together," Russell said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags managementTraffic managementapplicationsNetworkingsplunkdata miningsoftwareData management

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Essentials

Lexar® JumpDrive® S57 USB 3.0 flash drive

Learn more >

Microsoft L5V-00027 Sculpt Ergonomic Keyboard Desktop

Learn more >

Mobile

Lexar® JumpDrive® S45 USB 3.0 flash drive 

Learn more >

Exec

Lexar® Professional 1800x microSDHC™/microSDXC™ UHS-II cards 

Learn more >

HD Pan/Tilt Wi-Fi Camera with Night Vision NC450

Learn more >

Audio-Technica ATH-ANC70 Noise Cancelling Headphones

Learn more >

Lexar® JumpDrive® C20c USB Type-C flash drive 

Learn more >

Budget

Back To Business Guide

Click for more ›

Most Popular Reviews

Latest News Articles

Resources

PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?