Harnessing AI for Rail Safety and Efficiency

Tracking the Untrackable

Railways consist of a lot of different components – sleepers, rails, safety welds, signs, and more. Knowing the precise location and condition of these assets is paramount for smooth operations. The traditional approach to locating and inspecting these assets was by looking at video of the tracks and checking what assets were seen, which was manually intensive and time-consuming. To streamline the process, ProRail employed a "videoschouwtrein" (video inspection train) equipped to capture line scans and videos from the tracks using multiple perspectives. Yet, manually analyzing this footage was still a mammoth task. Enter AI.

AI to the Rescue

Our solution? Together with different suppliers at ProRail and ProRail themselves, we developed several deep learning models specialized to recognize specific assets: sleepers (rectangular supports that lie perpendicular to the rails), signs, and "Elektrische Scheidingslassen" (Electrical Separation Welds). These models could sift through vast amounts of video data, identifying assets with a precision that matched human levels, but at an exponentially faster rate.

The process, however, wasn't without its challenges. Linking the metadata from the inspection train to the images was complex. We had to synergize GPS signals, odometer readings, the train's route, and GIS maps from ProRail to accurately estimate the assets' locations. This was crucial not only for updating the existing asset database but also for identifying any potentially overlooked assets, especially when they were in close proximity to one another. Through meticulous application of business rules and AI, we could make recommendations on asset statuses, substantially reducing manual labor and the margin for error.

Assessing the Health of the Rails

Knowing where an asset is, is one thing; understanding its condition is another. This was particularly important for "ES-lassen" (Electrical Separation Welds), as they play a vital role in signaling by interrupting the electrical current to detect train presence. ProRail’s ambition is to provide detailed information on the condition of these welds. Traditional inspection methods were manual and hard to scale across the country, prompting the need for an automated solution.

Our focus was on a port area, a heavy freight corridor where the likelihood of damage to ES welds is notably high. At the Kijfhoek yard, the high density of trains occupying the rails made manual checks even more cumbersome. Our task was to explore the potential for automated image recognition to detect and assess damage to these welds systematically.

The project dealt with two types of damage: burr formation, which disrupts the insulating function of the welds, and battered rail heads, a precursor to more severe defects. Using video inspection footage and the previously developed AI model for ES weld detection, we were able to pre-select relevant images for further examination.

The Findings and Beyond

Our results were promising. The AI model proved capable of detecting both types of damage, allowing for a swift selection of images showing the most significant deterioration. This selection enabled inspectors to pinpoint damaged welds efficiently. We proposed delivering an Excel sheet post-inspection or using color-coded maps to display damage intensity, making the information accessible and actionable. In the analysis, 1,118 images of welds were found in a particular area. It was observed that a certain degree of wear was present in some of the images, which is a normal aspect of infrastructure use and does not imply immediate safety concerns or the need for replacement. In fact, 53% of the images indicated no significant issues, allowing inspectors to focus their attention more efficiently on the remaining 47%. With this process, inspectors are required to review approximately half as many images as before, significantly streamlining their workflow. Considering that an inspector can evaluate about five images per minute, this new method translates into a substantial time saving, further emphasizing the effectiveness of ProRail’s maintenance and inspection protocols.

However, not all damage could be unambiguously determined from the images. For example, when assessing the condition of a head with impact damage, it's crucial to measure its depth to determine whether it constitutes a defect. The depth measurement is a standard criterion used to identify defects, as it's not possible to make this assessment based solely on visual inspection. We therefore incorporated guidelines from stakeholders to define the condition parameters more clearly, such as using circular annotations as indicators.

The model was able to generalize to new areas, assuming the new images shared characteristics with the training set. This suggests that with consistent imaging properties, assured through quality checks of the video material, our solution could be scaled nationwide.

In conclusion, our project with ProRail underscores the potential of AI in enhancing the resilience and safety of railway operations. By applying our expertise in image recognition and machine learning, we've paved the way for a more automated future in railway maintenance and inspection, ensuring the safety of the Dutch rail network while minimizing disruptions.

At BigData Republic we are excited to continue this journey, pushing the boundaries of what's possible with AI in real-world applications. Stay tuned as we ride into a future where innovation meets infrastructure, head-on.