Zhou, K., Li, J., Xiao, Y., Yang, J., Cheng, J., Liu, W., Luo, W., Liu, J., & Gao, S. (2022). Memorizing Structure-Texture Correspondence for Image Anomaly Detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6), 2335–2349. https://doi.org/10.1109/tnnls.2021.3101403
This work focuses on image anomaly detection by leveraging only normal images in the training phase. Most
previous methods tackle anomaly detection by reconstructing the input images with an Auto-Encoder based model, and an underlying assumption is that the reconstruction errors for the normal images are small and those for the abnormal images are large. However, these Auto-Encoder based methods sometimes even reconstruct the anomalies well; consequently, they are less sensitive to anomalies. To conquer this issue, we propose to reconstruct the image by leveraging the structure-texture correspondence. Specifically, we observe that usually for normal
images, the texture can be inferred from its corresponding structure (e.g., the blood vessels in the fundus image and the structured anatomy in optical coherence tomography image), while it is hard to infer the texture from a destroyed structure for the abnormal images. is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture. As the correspondence between destroyed structure and texture cannot be characterized by the
memory, the abnormal images would have a larger reconstruction error, facilitating anomaly detection. In this work, we utilize two kinds of complementary structures (i.e., the semantic structure with human-labeled category information and the low-level structure with abundant details), which are extracted by two structure extractors. The reconstructions from the two kinds of structures are fused together by a learned attention weight to get the final reconstructed image. We further feed the reconstructed image into the two aforementioned structure extractors to extract structures. On the one hand, constraining the consistency between the structures extracted from the original input and that from the reconstructed image would regularize the network training; on the other hand, the error between the structures extracted from the original input and that from the reconstructed image can also be used as a supplement measurement to identify the anomaly. Extensive experiments validate the effectiveness of our method for image anomaly detection on both industrial inspection images and medical images.
The work was supported by National Key R&D Program
of China (2018AAA0100704), NSFC #61932020, Science
and Technology Commission of Shanghai Municipality (Grant
No. 20ZR1436000), Guangdong Provincial Department of
Education (2020ZDZX3043), Shenzhen Natural Science Fund
(JCYJ20200109140820699), the Stable Support Plan Program
(20200925174052004), and “Shuguang Program” supported
by Shanghai Education Development Foundation and Shanghai Municipal Education Commission.