NPR deploys Splunk for Web analytics

Using Splunk, NPR has found a faster way to build user data statistics

In a novel use of the software, National Public Radio is using the Splunk log search engine to analyze Web traffic for its audio streams and downloads.

NPR metrics analyst Sondra Russell described the setup at GigaOm's Big Data conference, held Wednesday in New York.

Splunk offers what it calls a search engine for machine data. It was originally built to parse log files, or the files programs and hardware generate to document their transactions, errors and other operational information. By coordinating the timestamps of messages from different applications and hardware, Splunk allows system administrators to pinpoint difficult-to-locate system problems.

In recent years, however, customers have been expanding their uses of Splunk to other duties, explained Splunk Chief Technology Officer Erik Swan, also speaking at the event. Web traffic analysis and business intelligence are two such ancillary uses.

For much of its Web traffic monitoring, NPR uses standard Web traffic analytic software, which can deliver reports on how many people visit each Web page. Such software usually generates these counts by using cookies or by embedding each page with a small script that alerts the software when the page is rendered in a browser.

The media organization, however, found it difficult to get reliable usage summaries for a number of aspects of its service. For instance, the organization needed to get an accurate count of how many listeners tuned into their streamed audio and video programs.

To get this data, NPR had prepared a PHP script that would parse the server log files and translate the results into a form that could be digested by Adobe's Omniture, a Web analytic tool. Getting information back, however, could take up to 24 hours, and Russell still didn't trust that the results provided an accurate count.

In the cases of streaming usage, many users might start a stream, then pause it, and restart it. Or perhaps a user would restart a stream after a failed Internet connection. In the server log files, all these events were logged as separate events, not a linear sequence of actions by a single user. As a result, there was no way of determining how many connections were from different listeners, and how many were multiple streams to a single user.

"With our PHP scripts, we could not get that level of sophistication. So our numbers could be off," Russell said."It was hard to make rational decisions based on this."

By working with Splunk, NPR could derive listener numbers and information directly from its servers' log files. The software allows users to script search results and then graph the results, or show them on a dashboard.

In NPR's case, for instance, it provided the exact number of listeners for each program that was streamed or downloaded. "With Splunk you can wrap multiple [log entries] into a single distinct visit, so I feel my numbers are actually accurate," she said.

Splunk helped identify users' mobile platforms as well. An increasing amount of traffic to the NPR site comes from mobile clients, such as iPhones, iPads and Android smartphones. In one case, a manager wanted to know which version of the iPhone operating system was most often used, as the results would direct the company's design work for its iPhone app.

"I could not have answered that without Splunk," Russell said.

Splunk also solved a seemingly unsolvable problem for the organization: determining how to pay royalties for streamed songs. NPR offers a streaming service for songs, called SoundExchange. It must pay out royalties for each song played, based on the number of listeners that stream had at the moment. "It can get complicated to reconcile the two," Russell said.

Using Splunk, Russell was able to merge two files -- a list of when each song was played, and the number of listeners that stream had when the song was played. "Splunk put them together," Russell said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Tags managementTraffic managementapplicationsNetworkingsplunksoftwaredata miningData management

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service

Comments

Comments are now closed.

Most Popular Reviews

Follow Us

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Latest News Articles

Resources

Best Deals on GoodGearGuide

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?