Autors: Shaikh M.Z., Mehran S., Baro E.N., Manolova, A. H., Uqaili M.A., Hussain T., Chowdhry B.S.
Title: Design and Development of a Wayside AI-Assisted Vision System for Online Train Wheel Inspection
Keywords: automated inspection system, deep learning, online data acquisition, YOLO

Abstract: Engineering Reports published by John Wiley & Sons Ltd.Wayside inspection of rolling stock has been around for some time and wheel impact load and fiber-grating sensors are actively explored for getting high-fidelity data. Visual inspection from wayside provides the opportunity to gain high-resolution data, which can help in the early diagnosis of potential faults. It is rarely explored due to complexities associated with calibration, moving and rotating targets, and difficulties associated with data acquisition. This paper explores and presents an in-depth design and development strategy for such systems. It presents the development steps, implementation, and results of a vision inspection system for regular and automated inspection of train wheels. First, various configurations for positioning of the cameras in a three-dimensional setting are considered and discussed, followed by online data acquisition for establishing a data set. Later, a comprehensive comparative analysis was conducted on several object detection algorithms for wheel segmentation task. Different algorithms are evaluated using COCO evaluation metrics, and the best-performing model, YOLOv9, achieves a mAP50 of 0.94 and a recall of 0.91. The developed system has produced satisfactory results in acquiring proper wheel tread images and segmenting the wheel. Further avenues for countering lighting issues and defect detection are provided.

References

  1. S. Y. Chong, J.-R. Lee, and H.-J. Shin, “A Review of Health and Operation Monitoring Technologies for Trains,” Smart Structures and Systems 6, no. 9 (2010): 1079–1105, https://doi.org/10.12989/sss.2010.6.9.1079.
  2. A. Johansson and J. C. O. Nielsen, “Out-Of-Round Railway Wheels—Wheel-Rail Contact Forces and Track Response Derived From Field Tests and Numerical Simulations,” Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit 217, no. 2 (2003): 135–146.
  3. R. W. Ngigi, C. Pislaru, A. Ball, and F. Gu, “Modern Techniques for Condition Monitoring of Railway Vehicle Dynamics,” Journal of Physics Conference Series 364, no. 1 (2012), https://doi.org/10.1088/1742-6596/364/1/012016.
  4. A. Alemi, F. Corman, and G. Lodewijks, “Condition Monitoring Approaches for the Detection of Railway Wheel Defects,” Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit 231, no. 8 (2017): 961–981, https://doi.org/10.1177/0954409716656218.
  5. S. Lv, F. Zhou, and Z. Wei, “Train Wheel Tread Defects Detection Based on Image Registration,” in IST 2017—IEEE International Conference on Imaging Systems and Techniques, Proceedings, vol. 2018 (Beijing, China: IEEE, 2017), 1–4, https://doi.org/10.1109/IST.2017.8261509.
  6. G. Guo, J. Peng, K. Yang, L. Xie, and W. Song, “Wheel Tread Defects Inspection Based on SVM,” in Proceedings of 2017 IEEE Far East NDT New Technology and Application Forum, FENDT 2017 (Beijing, China: IEEE, 2018), 251–253, https://doi.org/10.1109/FENDT.2017.8584565.
  7. A. Trilla and X. Cabré, “Determining the Equivalent Conicity for Railway Wheelset Maintenance With Deep Ensembles,” in Proceedings of the Annual Conference of the Prognostics and Health Management Society, PHM (Philadelphia, Pennsylvania: IEEE, 2018), 1–6, https://doi.org/10.36001/phmconf.2018.v10i1.465.
  8. G. Krummenacher, C. S. Ong, S. Koller, S. Kobayashi, and J. M. Buhmann, “Wheel Defect Detection With Machine Learning,” IEEE Transactions on Intelligent Transportation Systems 19, no. 4 (2018): 1176–1187, https://doi.org/10.1109/TITS.2017.2720721.
  9. M. Z. Shaikh, Z. Ahmed, B. S. Chowdhry, et al., “State-Of-The-Art Wayside Condition Monitoring Systems for Railway Wheels: A Comprehensive Review,” IEEE Access 11 (2023): 13257–13279, https://doi.org/10.1109/ACCESS.2023.3240167.
  10. M. de Almeida Costa, J. P. de Azevedo Peixoto Braga, and A. R. Andrade, “Assessing the Performance of Different Devices in Railway Wheelset Inspection,” Measurement 165 (2020): 108145, https://doi.org/10.1016/j.measurement.2020.108145.
  11. M. Mohammadi, A. Mosleh, C. Vale, D. Ribeiro, P. Montenegro, and A. Meixedo, “An Unsupervised Learning Approach for Wayside Train Wheel Flat Detection,” Sensors 23, no. 4 (2023), https://doi.org/10.3390/s23041910.
  12. M. Z. Shaikh, Z. Ahmed, E. N. Baro, S. Hussain, and M. Milanova, “Deep Learning Based Identification and Tracking of Railway Bogie Parts,” Alexandria Engineering Journal 107 (2024): 533–546, https://doi.org/10.1016/J.AEJ.2024.07.064.
  13. A. Lourenço, C. Ferraz, D. Ribeiro, et al., “Adaptive Time Series Representation for Out-Of-Round Railway Wheels Fault Diagnosis in Wayside Monitoring,” Engineering Failure Analysis 152 (2023): 107433, https://doi.org/10.1016/j.engfailanal.2023.107433.
  14. J. Magalhães, T. Jorge, R. Silva, et al., “A Strategy for Out-Of-Roundness Damage Wheels Identification in Railway Vehicles Based on Sparse Autoencoders,” Railway Engineering Science 1 (2024): 1–23, https://doi.org/10.1007/s40534-024-00338-4.
  15. T. Jorge, J. Magalhães, R. Silva, et al., “Early Identification of Out-Of-Roundness Damage Wheels in Railway Freight Vehicles Using a Wayside System and a Stacked Sparse Autoencoder,” Vehicle System Dynamics (2024): 1–26, https://doi.org/10.1080/00423114.2024.2333771.
  16. A. Trilla, J. Bob-Manuel, B. Lamoureux, and X. Vilasis-Cardona, “Integrated Multiple-Defect Detection and Evaluation of Rail Wheel Tread Images Using Convolutional Neural Networks,” International Journal of Prognostics and Health Management 12, no. 1 (2021), https://doi.org/10.36001/ijphm.2021.v12i1.2906.
  17. A. Guedes, R. Silva, D. Ribeiro, et al., “Clustering-Based Classification of Polygonal Wheels in a Railway Freight Vehicle Using a Wayside System,” Applied Sciences 14, no. 9 (2024), https://doi.org/10.3390/app14093650.
  18. X. Z. Liu and Y. Q. Ni, “Wheel Tread Defect Detection for High-Speed Trains Using FBG-Based Online Monitoring Techniques,” Smart Structures and Systems 21, no. 5 (2018): 687–694, https://doi.org/10.12989/sss.2018.21.5.687.
  19. X. Z. Liu, C. Xu, and Y. Q. Ni, “Wayside Detection of Wheel Minor Defects in High-Speed Trains by a Bayesian Blind Source Separation Method,” Sensors 19, no. 18 (2019), https://doi.org/10.3390/s19183981.
  20. Y. W. Wang, Y. Q. Ni, and X. Wang, “Real-Time Defect Detection of High-Speed Train Wheels by Using Bayesian Forecasting and Dynamic Model,” Mechanical Systems and Signal Processing 139 (2020): 106654, https://doi.org/10.1016/j.ymssp.2020.106654.
  21. Y. Q. Ni and Q. H. Zhang, “A Bayesian Machine Learning Approach for Online Detection of Railway Wheel Defects Using Track-Side Monitoring,” Structural Health Monitoring 20 (2020): 1536–1550, https://doi.org/10.1177/1475921720921772.
  22. P. Hyde, F. Defossez, and C. Ulianov, “Development and Testing of an Automatic Remote Condition Monitoring System for Train Wheels,” IET Intelligent Transport Systems 10, no. 1 (2016): 32–40, https://doi.org/10.1049/iet-its.2015.0041.
  23. C.-Z. Dong and F. N. Catbas, “A Review of Computer Vision–Based Structural Health Monitoring at Local and Global Levels,” Structural Health Monitoring 20, no. 2 (2021): 692–743.
  24. M. Abadi, A. Agarwal, P. Barham, et al., “Tensorflow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” arXiv preprint arXiv:1603.04467, 2016.
  25. G. Jocher, A. Stoken, J. Borovec, et al., “ultralytics/yolov5: v5.0—YOLOv5-P6 1280 Models, AWS, Supervise.ly and YouTube Integrations,” 2021 https://doi.org/10.5281/ZENODO.4679653.
  26. K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian, “Centernet: Keypoint Triplets for Object Detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (South Korea: IEEE, 2019), 6569–6578.
  27. M. Tan, R. Pang, and Q. V. Le, “EfficientDet: Scalable and Efficient Object Detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Seattle, WA: IEEE, 2019), 10778–10787, https://doi.org/10.1109/CVPR42600.2020.01079.
  28. R. Girshick, “Fast R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision (Santiago, Chile: IEEE, 2015), 1440–1448.
  29. C. Cheng, “Real-Time Mask Detection Based on SSD-MobileNetV2,” in 2022 IEEE 5th International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) (Shenyang, China: IEEE, 2022), 761–767, https://doi.org/10.1109/AUTEEE56487.2022.9994442.

Issue

Engineering Reports, 2024, , https://doi.org/10.1002/eng2.13027

Цитирания (Citation/s):
1. Ye, Yunguang, et al. "Fault diagnosis of railway wheelsets: A review." Measurement (2024): 116169. https://doi.org/10.1016/j.measurement.2024.116169 - 2024 - в издания, индексирани в Scopus
2. Lee S., Choi J., Kim N., Hyeon J., Real-time Vision-based Wheel-Rail Interface Monitoring for Tramway Running Performance, 2025, IEEE Access, issue 0, DOI 10.1109/ACCESS.2025.3585740, eissn 21693536 - 2025 - в издания, индексирани в Scopus
3. Cascino A., Delle Monache S., Lanzillo L., Mazzeo F., Nencioni L., Nicolella A., Strano S., Terzo M., Development and Experimental Testing of a 3D Vision System for Railway Freight Wagon Monitoring, 2025, Applied Sciences Switzerland, issue 21, vol. 15, DOI 10.3390/app152111547, eissn 20763417 - 2025 - в издания, индексирани в Scopus
4. Gao M., Yang K., Qi M., Meng A., Guo X., TEDS EMU Fault Image Dataset, 2025, Engineering Reports, issue 12, vol. 7, DOI 10.1002/eng2.70447, eissn 25778196 - 2025 - в издания, индексирани в Scopus
5. Dudek A., Patalas-Maliszewska J., Analysis of the impact of the interferences in the vision system on a collision detection in the HRC workstation towards Industry 5.0, 2025, IFAC Papersonline, issue 24, vol. 59, pp. 327-332, DOI 10.1016/j.ifacol.2025.11.886, issn 24058971, eissn 24058963 - 2025 - в издания, индексирани в Scopus
6. Askar S., Saidoune F., Abduvalieva D., Ezhilarasan G., Thakur R., Yadav Y., Kumar R., Hussen M., Jawad I.A., Ali A.M., Stamov G., Stamov G., Darvishpoor S., Roshanian J., Mesbah A., Haghighi K., Ebrahimi B., Shaikh M.Z., Jatoi S., Baro E.N., Das B., Hussain S., Chowdhry B.S., FaultSeg: A Dataset for Train Wheel Defect Detection, 2025, Scientific Datavol. 12, , DOI omid:br/0605789451 doi:10.1038/s41597-025-04557-0 - 2025 - в издания, индексирани в Scopus
7. Kiryakova V., Li C., Wang P., Long M., Xu Y., Han R., Jia T., Han T., Chen P., Zhang Y., Tang S.J., Teng M.Y., Shi C.Y., Ye L.W., Huang G.L., Chen Z., Han H.B., Lu G., Zheng Y.X., Popova M., Mladenov B., Dimitrov I., Dimitrov M., Mitova V., Mitrev Y., Kovacheva D., Velinov N., Karashanova D., Medina Y.G., Flores R., Radoslavov A., Svezhenov Y., Contreras-Ferreyra D.U., Martínez-Rojas A., Pastor-Gómez N., Chávez-Negrete C., Popova M., Mladenov B., Dimitrov I., Dimitrov M., Mitova V., Mitrev Y., Kovacheva D., Velinov N., Karashanova D., Reyes V.A., De Leon Ramirez J.I., Perez-Cabrera L., Estolano-Cobian A., Chavez-Mendez J.R., Petranovskii V., Jansson E., Scheuer V., Jordan E., Kostourou K., Mehlstäubler T.E., Wang Y., Yang Z., Wu J., Ma X., Zhou L., Li X., Ma B., Qiu Z., Kang L., Celik H.K., Cinar R., Rennie A.E.W., Zahariev A., Kashyap N., Kashyap A.K., Varshney A., Patial S.K., Sharma A., Singh S., Kumar B., Jangra S., Das S., Goyat M.S., Parkhomenko R.G., De Luca O., Rudolf P., Modin E., Martínez D.M., Cunha L., Knez M., Iliev R., Madzharov N., Gilev B., Stošić B.P., Dimitrijević M.S., Lu D., Jin K., Shen Y., Kotlarski G., Stoeva D., Dechev D., Ivanov N., Ormanova M., Yotov I., Todorov G., Gavrilov T., Guozhong H., Wenzhao L., Xuehong G., Lingyun X., Yan W., Honglei D., Kumar V., Singh H., Tsenov G.T., Popova M., Mladenov B., Dimitrov I., Dimitrov M., Mitova V., Mitrev Y., Kovacheva D., Velinov N., Karashanova D., Kotsiopoulos T., Radoglou-Grammatikis P., Lekka Z., Kuru K., Kuru K., Vasileva K., Yordanov Y., Manasiev Y., Stefanova D., Kondeva-Burdina M., Tzankova V., Panev T., Madzharov N., Gilev B., Bozhilov I.B., Vasileva K., Yordanov Y., Manasiev Y., Stefanova D., Kondeva-Burdina M., Tzankova V., Panev T., Ivanov V., Melenchuk T., Ivanova S., Karaivanov D., Volkova M., Guliashki V., Kotsiopoulos T., Radoglou-Grammatikis P., Lekka Z., Kanchev N., Sapundzhi F., Dimitrova V., Serrano-Febles J., Luis-Orozco J., Luis Ramírez H., Gómez-Brizuela L., Martín-Medina C., García-Cruz A., Madzharov N., Gilev B., Stoeva D., Kotlarski G., Dechev D., Ivanov N., Ormanova M., Chowdhury S., Ghosh A., Acharya S., Saha S., Pal P.K., Roy S., Lahiri S.K., Becil E.J., Jimenez A.J., Cortes D.A.R., Salgado R.G., Alebicto M.E., Todorov T., Soforonov Y., Wang J., Hu Y., Dong X., Tao J., Yan B., Guo W., Chen G., Kiryakova V., Kaur N., Ghalwan M., Kumar K., Subramanian S.S., Souja R., Tsenov G.T., Tsenov G.T., Borissova A., Tomov G., Genova T., Zhelyazkova A., Zaharieva L., Angelov I., Budakova D., Banchelli L., Stavrov V., Gotseva N., Vlahov A., Mfondoum R., Ivanov A., Zylfiu B., Muka E., Mankolli E., Guliashki V., Darvishpoor S., Roshanian J., Mesbah A., Haghighi K., Ebrahimi B., Dimitrijević M.S., Popa L.D., Isticioaia S.F., Vlăduț N.V., Matei G., Leonte A., Găgeanu I., Nenciu F., Pintilie P.L., Pintilie A.S., Milea O., Roșca M., Panda A., Teliban G.C., Burducea M., Gatev V., Dimitrijević M.S., Abro B., Jatoi S., Shaikh M.Z., Baro E.N., Chowdhry B.S., Milanova M., Towards Smarter Road Maintenance: YOLOv7-Seg for Real-Time Detection of Surface Defects, 2025, Lecture Notes in Computer Sciencevol. 15618 LNCS, pp. 36-54, , DOI omid:br/06011115271 doi:10.1007/978-3-031-88220-3_3 - 2025 - в издания, индексирани в Scopus

Вид: статия в списание, публикация в издание с импакт фактор, публикация в реферирано издание, индексирана в Scopus