“TfNSW needed a solution to capture real-time data for every vehicle in motion across the state. This solution just gives us that so that we mine nuggets from this data at a later date. We now have an ability to self-service without waiting for someone else to curate operational data.”
Historical GTFS data has always been too large and costly to store efficiently and analyze. Every TfNSW vehicle sends its position and other telemetry every 10 seconds, which results in a huge stockpile of data.
In the past, this GTFS Realtime data was not retained; at any given time, only the latest copy is published on the Transport Open Data Hub. This makes it impossible to obtain insights from past services, ruling out valuable insights such as the ability to predict trip delays, optimize trip routes, report, and analyze service performance, thereby improving customer services.
Key operational data-related challenges to overcome:
Modern data platform capable of ingesting and storing infinite operational data in realtime or batch in a well-governed, secure and cost-effective manner.
Real-time positions and telemetry for every TfNSW mode is now tracked and stored for analysis
Self-service analytics and data-sharing on those operational data sets can be done by internal and public users
“With the ODL platform we are able to ingest and process 500GB, millions of various data files a day, in real time and batch efficiently, which is unprecedented in NSW Transport. The ODL is a great example of building a next-generation Cloud-based data and analytics platform using native Azure services. We can deliver what we had in mind with the Azure ODL because it is a flexible, rich in services and features, high-performant and easily extensible.”
