Hadoop analysis now tackling IoT to improve transit

A new consultancy crunches data from different sources for insights into agencies' operations

For the Internet of Things, collecting data is only half the battle, and public transit agencies know that as well as anyone.

Operators of bus, train and trolley systems collect a wide range of data types from ticket machines, fare validation devices, GPS (Global Positioning System) and other sources, but typically that data stays in silos, according to Wade Rosado, director of analytics for Urban Insights, a consultancy launched on Monday to help make use of transit data.

Urban Insights is using a cloud-based Apache Hadoop system to crunch numbers from transit agencies that want to glean new insights about their operations. The company, a wholly owned subsidiary of transit technology vendor Cubic Transportation Systems, can provide both periodic consulting services, such as producing annual reviews, and ongoing measurements as frequently as every day, Rosado said.

Transit is a classic example of the kind of business that may be transformed by IoT. It has a lot of moving parts, serves constantly changing rider needs and faces demands for efficiency. Transit also is built around specialized infrastructure that stays in place for years or even decades, something it holds in common with power transmission, logistics and other IoT hotspots.

There's a lot of useful data generated in a transit system's day, Rosado said. Buses may check in with a GPS location every 30 seconds, sensors on doors can count how many riders get on and off, and payment systems know about ticket purchases and validations.

"They do absolutely collect this data, and they use it within the constraints of the system that was designed to report on it, but where the weakness exists is in integrating those data sources to paint a more complete picture of what's actually happening," Rosado said.

For example, Urban Insights is helping the San Diego Municipal Transportation System (MTS) to figure out whether its routes and schedules match most riders' needs. Existing systems collected data about individual trips on buses or trolleys, so Urban Insights combined data from five separate sources to understand riders' overall journeys, including transfers, and whether service was aligned with real demand, he said.

Urban Insights creates custom data models, takes in raw data from customers' systems and reformats it, and then integrates it with data from other sources. "Hadoop lends itself very well to that, because it's able to do complex operations on very large data sets," Rosado said.

Urban Insights, based in Washington, D.C., is running its service on a combination of its own servers and virtual private cloud resources through services like Microsoft Azure, Rosado said. That lets the consultancy burst to greater capacity when needed, he said.

The transit industry is moving toward greater integration, but that's easiest for large cities, Rosado said. For example, Cubic Transportation Systems built a platform for the Chicago Transit Authority that combines CRM (customer relationship management), financial settlements, fair payments, cash management and other functions. But smaller agencies are often limited to off-the-shelf products that don't bring as much data together in one place, he said.

There are also emerging standards that span different types of data, such as GTFS (General Transit Feed Specification), a common format for public transit schedules and location information that can be used with the Google Transit planning tool. But it's still early days, Rosado said.

Stephen Lawson covers mobile, storage and networking technologies for The IDG News Service. Follow Stephen on Twitter at @sdlawsonmedia. Stephen's e-mail address is stephen_lawson@idg.com

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags Government use of ITCubic Transportation SystemsUrban Insightsgovernmentinternet

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Stephen Lawson

IDG News Service
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

David Coyle

Brother PocketJet PJ-773 A4 Portable Thermal Printer

I rate the printer as a 5 out of 5 stars as it has been able to fit seamlessly into my busy and mobile lifestyle.

Kurt Hegetschweiler

Brother PocketJet PJ-773 A4 Portable Thermal Printer

It’s perfect for mobile workers. Just take it out — it’s small enough to sit anywhere — turn it on, load a sheet of paper, and start printing.

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?