Memorizing Structure-Texture Correspondence for Image Anomaly Detection

Page view(s)
103
Checked on Oct 22, 2024
Memorizing Structure-Texture Correspondence for Image Anomaly Detection
Title:
Memorizing Structure-Texture Correspondence for Image Anomaly Detection
Journal Title:
IEEE Transactions on Neural Networks and Learning Systems
Publication Date:
13 August 2021
Citation:
Zhou, K., Li, J., Xiao, Y., Yang, J., Cheng, J., Liu, W., Luo, W., Liu, J., & Gao, S. (2022). Memorizing Structure-Texture Correspondence for Image Anomaly Detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6), 2335–2349. https://doi.org/10.1109/tnnls.2021.3101403
Abstract:
This work focuses on image anomaly detection by leveraging only normal images in the training phase. Most previous methods tackle anomaly detection by reconstructing the input images with an Auto-Encoder based model, and an underlying assumption is that the reconstruction errors for the normal images are small and those for the abnormal images are large. However, these Auto-Encoder based methods sometimes even reconstruct the anomalies well; consequently, they are less sensitive to anomalies. To conquer this issue, we propose to reconstruct the image by leveraging the structure-texture correspondence. Specifically, we observe that usually for normal images, the texture can be inferred from its corresponding structure (e.g., the blood vessels in the fundus image and the structured anatomy in optical coherence tomography image), while it is hard to infer the texture from a destroyed structure for the abnormal images. is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture. As the correspondence between destroyed structure and texture cannot be characterized by the memory, the abnormal images would have a larger reconstruction error, facilitating anomaly detection. In this work, we utilize two kinds of complementary structures (i.e., the semantic structure with human-labeled category information and the low-level structure with abundant details), which are extracted by two structure extractors. The reconstructions from the two kinds of structures are fused together by a learned attention weight to get the final reconstructed image. We further feed the reconstructed image into the two aforementioned structure extractors to extract structures. On the one hand, constraining the consistency between the structures extracted from the original input and that from the reconstructed image would regularize the network training; on the other hand, the error between the structures extracted from the original input and that from the reconstructed image can also be used as a supplement measurement to identify the anomaly. Extensive experiments validate the effectiveness of our method for image anomaly detection on both industrial inspection images and medical images.
License type:
Publisher Copyright
Funding Info:
The work was supported by National Key R&D Program of China (2018AAA0100704), NSFC #61932020, Science and Technology Commission of Shanghai Municipality (Grant No. 20ZR1436000), Guangdong Provincial Department of Education (2020ZDZX3043), Shenzhen Natural Science Fund (JCYJ20200109140820699), the Stable Support Plan Program (20200925174052004), and “Shuguang Program” supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission.
Description:
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
2162-2388
2162-237X
Files uploaded:

File Size Format Action
tnnls.pdf 2.24 MB PDF Open