Hadoop analysis now tackling IoT to improve transit

A new consultancy crunches data from different sources for insights into agencies' operations

For the Internet of Things, collecting data is only half the battle, and public transit agencies know that as well as anyone.

Operators of bus, train and trolley systems collect a wide range of data types from ticket machines, fare validation devices, GPS (Global Positioning System) and other sources, but typically that data stays in silos, according to Wade Rosado, director of analytics for Urban Insights, a consultancy launched on Monday to help make use of transit data.

Urban Insights is using a cloud-based Apache Hadoop system to crunch numbers from transit agencies that want to glean new insights about their operations. The company, a wholly owned subsidiary of transit technology vendor Cubic Transportation Systems, can provide both periodic consulting services, such as producing annual reviews, and ongoing measurements as frequently as every day, Rosado said.

Transit is a classic example of the kind of business that may be transformed by IoT. It has a lot of moving parts, serves constantly changing rider needs and faces demands for efficiency. Transit also is built around specialized infrastructure that stays in place for years or even decades, something it holds in common with power transmission, logistics and other IoT hotspots.

There's a lot of useful data generated in a transit system's day, Rosado said. Buses may check in with a GPS location every 30 seconds, sensors on doors can count how many riders get on and off, and payment systems know about ticket purchases and validations.

"They do absolutely collect this data, and they use it within the constraints of the system that was designed to report on it, but where the weakness exists is in integrating those data sources to paint a more complete picture of what's actually happening," Rosado said.

For example, Urban Insights is helping the San Diego Municipal Transportation System (MTS) to figure out whether its routes and schedules match most riders' needs. Existing systems collected data about individual trips on buses or trolleys, so Urban Insights combined data from five separate sources to understand riders' overall journeys, including transfers, and whether service was aligned with real demand, he said.

Urban Insights creates custom data models, takes in raw data from customers' systems and reformats it, and then integrates it with data from other sources. "Hadoop lends itself very well to that, because it's able to do complex operations on very large data sets," Rosado said.

Urban Insights, based in Washington, D.C., is running its service on a combination of its own servers and virtual private cloud resources through services like Microsoft Azure, Rosado said. That lets the consultancy burst to greater capacity when needed, he said.

The transit industry is moving toward greater integration, but that's easiest for large cities, Rosado said. For example, Cubic Transportation Systems built a platform for the Chicago Transit Authority that combines CRM (customer relationship management), financial settlements, fair payments, cash management and other functions. But smaller agencies are often limited to off-the-shelf products that don't bring as much data together in one place, he said.

There are also emerging standards that span different types of data, such as GTFS (General Transit Feed Specification), a common format for public transit schedules and location information that can be used with the Google Transit planning tool. But it's still early days, Rosado said.

Stephen Lawson covers mobile, storage and networking technologies for The IDG News Service. Follow Stephen on Twitter at @sdlawsonmedia. Stephen's e-mail address is stephen_lawson@idg.com

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags Government use of ITCubic Transportation SystemsUrban Insightsgovernmentinternet

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Stephen Lawson

IDG News Service
Show Comments

Essentials

Lexar® JumpDrive® S57 USB 3.0 flash drive

Learn more >

Microsoft L5V-00027 Sculpt Ergonomic Keyboard Desktop

Learn more >

Mobile

Lexar® JumpDrive® S45 USB 3.0 flash drive 

Learn more >

Exec

Audio-Technica ATH-ANC70 Noise Cancelling Headphones

Learn more >

Lexar® JumpDrive® C20c USB Type-C flash drive 

Learn more >

HD Pan/Tilt Wi-Fi Camera with Night Vision NC450

Learn more >

Lexar® Professional 1800x microSDHC™/microSDXC™ UHS-II cards 

Learn more >

Budget

Back To Business Guide

Click for more ›

Most Popular Reviews

Latest News Articles

Resources

PCW Evaluation Team

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?