Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis

Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis

Cell Reports Medicine

10.1016/j.xcrm.2024.101876

https://doi.org/10.1016/j.xcrm.2024.101876

Yuanyuan Peng, Aidi Lin, Meng Wang, Tian Lin, Linna Liu, Jianhua Wu, Ke Zou, Tingkun Shi, Lixia Feng, Zhen Liang, Tao Li, Dan Liang, Shanshan Yu, Dawei Sun, Jing Luo, Ling Gao, Xinjian Chen, Binwei Huang, Chaoxin Zheng, Chuang Jin, Dezhi Zheng, Dingguo Huang, Dongjie Li, Guihua Zhang, Hanfu Wu, Honghe Xia, Hongjie Lin, Huiyu Liang, Jingsheng Yi, Jinqu Huang, Juntao Liu, Man Chen, Qin Zeng, Taiping Li, Weiqi Chen, Xia Huang, Xiaolin Chen, Xixuan Ke, Xulong Liao, Yifan Wang, Yin Huang, Yinglin Cheng, Yinling Zhang, Yongqun Xiong, Yuqiang Huang, Zhenggen Wu, Zijing Huang, Ching-Yu Cheng, Huazhu Fu, Haoyu Chen

19 December 2024

Peng, Y., Lin, A., Wang, M., Lin, T., Liu, L., Wu, J., Zou, K., Shi, T., Feng, L., Liang, Z., Li, T., Liang, D., Yu, S., Sun, D., Luo, J., Gao, L., Chen, X., Huang, B., Zheng, C., et al. (2025). Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis. Cell Reports Medicine, 6(1), 101876. https://doi.org/10.1016/j.xcrm.2024.101876

Inability to express the confidence level and detect unseen disease classes limits the clinical implementation of artificial intelligence in the real world. We develop a foundation model with uncertainty estimation (FMUE) to detect 16 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieves a higher F1 score of 95.74% than other state-of-the-art algorithms (92.03%–93.66%) and improves to 97.44% with threshold strategy. The model achieves similar excellent performance on two external test sets from the same and different OCT machines. In human-model comparison, FMUE achieves a higher F1 score of 96.30% than retinal experts (86.95%, p = 0.004), senior doctors (82.71%, p < 0.001), junior doctors (66.55%, p < 0.001), and generative pretrained transformer 4 with vision (GPT-4V) (32.39%, p < 0.001). Besides, FMUE predicts high uncertainty scores for >85% images of non-target-category diseases or with low quality to prompt manual checks and prevent misdiagnosis. Our FMUE provides a trustworthy method for automatic retinal anomaly detection in a clinical open-set environment.

Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

This research was supported by the National Key R&D Program of China (2018 YFA0701700 to H.C. and X.C.), Agency for Science, Technology and Research (A*STAR) Career Development Fund (C222812010 to H.F.), Central Research Fund (‘‘Robust and Trustworthy AI system for Multi-modality Healthcare’’ to H.F.), the National Nature Science Foundation of China (U20A20170 to X.C.), Shantou Science and Technology Program (190917085269835 to H.C.), Department of Education of Guangdong Province (2024ZDZX2024 to H.C.), and the University Natural Science Research Project of Anhui Province (2022AH040099 to Z.L. and 2023AH052070 to Y.P.).

https://oar.a-star.edu.sg/communities-collections/articles/23217

2666-3791

Institute of High Performance Computing