Exploring Structural Knowledge for Automated Visual Inspection of Moving Trains

Exploring Structural Knowledge for Automated Visual Inspection of Moving Trains
Exploring Structural Knowledge for Automated Visual Inspection of Moving Trains
Other Titles:
IEEE Transactions on Cybernetics
Publication Date:
19 June 2020
C. Chen, X. Zou, Z. Zeng, Z. Cheng, L. Zhang and S. C. H. Hoi, "Exploring Structural Knowledge for Automated Visual Inspection of Moving Trains," in IEEE Transactions on Cybernetics, doi: 10.1109/TCYB.2020.2998126.
Deep learning methods are becoming the de-facto standard for generic visual recognition in the literature. However, their adaptations to industrial scenarios, such as visual recognition for machines, product streamlines, etc., which consist of countless components, have not been investigated well yet. Compared with the generic object detection, there is some strong structural knowledge in these scenarios (e.g., fixed relative positions of components, component relationships, etc.). A case worth exploring could be automated visual inspection for trains, where there are various correlated components. However, the dominant object detection paradigm is limited by treating the visual features of each object region separately without considering common sense knowledge among objects. In this article, we propose a novel automated visual inspection framework for trains exploring structural knowledge for train component detection, which is called SKTCD. SKTCD is an end-to-end trainable framework, in which the visual features of train components and structural knowledge (including hierarchical scene contexts and spatial-aware component relationships) are jointly exploited for train component detection. We propose novel residual multiple gated recurrent units (Res-MGRUs) that can optimally fuse the visual features of train components and messages from the structural knowledge in a weighted-recurrent way. In order to verify the feasibility of SKTCD, a dataset that contains high-resolution images captured from moving trains has been collected, in which 18 590 critical train components are manually annotated. Extensive experiments on this dataset and on the PASCAL VOC dataset have demonstrated that SKTCD outperforms the existing challenging baselines significantly. The dataset as well as the source code can be downloaded online (https://github.com/smartprobe/SKCD).
License type:
Funding Info:
The work was supported by Singapore-China NRF-NSFC Grant, “Multi-Source Multi-Target Deep Neural Networks based on Novel Parallel/Distributed CPU/GPU Hybrid Architecture for Train Fault Detection”, No. NRF2016NRFNSFC001-111.
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Files uploaded:
File Size Format Action
There are no attached files.