Bring-Your-Own-Data for Network Rail
The second case study addresses Network Rail’s problem and vision statements. Network Rail provided fresh challenges for the established end-to-end data capture and processing system to be able to automate the ingestion, linear referencing and matching of LiDAR scans from different vendors. The Bring-your-own-Data scenario effectively demands more front-end processing to pre-process disparate data into the established automated process (shown in Figure 6 below).
FIGURE 6 The core Automated Data Processing Platform as deployed on ARTC, which was built out for the Bring-your-own-Data scenario for Network Rail
Network Rail’s system – scale of the solution requirement
In addition to enabling the Bring-your-own-Data scenario, the key to the success of the second case study was the ability to scale the solution to the volume of data processing requirements. Network Rail spans 15,904 route km (approximately 10,000 route miles, and 20,000 track miles) and 245,000 structures, serving 2,567 stations and over 10 billion passenger miles per quarter on a tightly constrained infrastructure, which was built piecemeal to different standards since the 1830s. In 2018/19, the network delivered 1.8 billion passenger journeys (on 250 different types of rail vehicles requiring clearance) and 17.4 billion net tonne-km (11.9 billion ton-miles) of freight.
Variation Between Scanned Datasets – Resolving the Data Integration Bottleneck
In reality, although LiDAR scanning data capture suppliers are all provided with an exacting specification by Network Rail, significant differences were found between successive data capture runs, with variation between individual operators working for the same vendor with the same equipment and outputting the same file type, as well as between vendors. Tiles were found missing or overlapping at interfaces between runs and different fields were incompletely populated. Going beyond the remit during development, Cordel provided free consultancy advice as invited, to help Network Rail to future-proof more consistency for future data capture, by aiding the development of guidance to set out more detail on how to apply Network Rail’s specifications. Meanwhile, new algorithms were built to accommodate the variation found, automating the pre-processing of the various datasets to flow into the established automated data pipeline.
Ingesting Mixed Vendor Data at Scale and Resolving Issues with Linear Referencing Systems
The sheer volume of data captured by the existing vendors presents challenges even for Network Rail to simply transfer data within the railroad, let alone share with others. An early milestone was to create and implement an uploader that gives Network Rail the ability to input LAZ files and trajectory data at-desktop (accessing Cordel’s cloud storage backend) enabling the linear reference system to be structured across the data for the entire network. Data importation from the range of different LiDAR data capture sources followed, with out-of-the-box compatibility demonstrated by running capacity tests across the distributed processing array.
Creating 2D Cross-sections and applying Machine Learning for transit space monitoring
The LiDAR point clouds were rendered into 2D cross sections at approximately 1-metre (3’) intervals, five times more frequent (and therefore 5 times more precise) than current requirements incorporating manual processing. The joining and overlaying of run data, coupled with HD imagery and sophisticated functions built out from GPS and dead-reckoning, together with this greater granularity, all contribute to better accuracy than previous manual systems. All this has enabled the resolution of anomalies within the National Gauging Database, effectively over-ruling the accepted version of the truth. Network Rail now sees the opportunity to create feedback loops (albeit wisely checked for engineering integrity by retaining a Human-in-the-Loop for critical decision-making), enabling automated processing to radically improve output integrity, despite the variable quality of scanned input data.
FIGURE 7 Stages in automating the processing of LiDAR scan data to derive railroad-specific insights from transit space monitoring, starting with the LRS (Linear Reference System)
Next the established suite of automated gauging processing is applied, as tailored to the precise requirements of the particular railroad, i.e. Network Rail. Transit space monitoring serves a range of specifications and regulatory requirements for users across the railroad – and delivering the solution sought-after in Network Rail’s Targets, as set in Figure 2 above
