Enhancing diagnostic accuracy in rare and common fundus diseases with a knowledge-rich vision-language model

Enhancing diagnostic accuracy in rare and common fundus diseases with a knowledge-rich vision-language model

Nature Communications

10.1038/s41467-025-60577-9

https://doi.org/10.1038/s41467-025-60577-9

Meng Wang, Tian Lin, Aidi Lin, Kai Yu, Yuanyuan Peng, Lianyu Wang, Cheng Chen, Ke Zou, Huiyu Liang, Man Chen, Xue Yao, Meiqin Zhang, Binwei Huang, Chaoxin Zheng, Peixin Zhang, Wei Chen, Yilong Luo, Yifan Chen, Honghe Xia, Tingkun Shi, Qi Zhang, Jinming Guo, Xiaolin Chen, Jingcheng Wang, Yih Chung Tham, Dianbo Liu, Wendy Wong, Sahil Thakur, Beau J. Fenner, Danqi Fang, Siying Liu, Qingyun Liu, Yuqiang Huang, Hongqiang Zeng, Yanda Meng, Yukun Zhou, Zehua Jiang, Minghui Qiu, Changqing Zhang, Xinjian Chen, Sophia Y. Wang, Cecilia S. Lee, Lucia Sobrin, Carol Y. Cheung, Chi Pui Pang, Pearse A. Keane, Ching-Yu Cheng, Haoyu Chen, Huazhu Fu

01 July 2025

Wang, M., Lin, T., Lin, A., Yu, K., Peng, Y., Wang, L., Chen, C., Zou, K., Liang, H., Chen, M., Yao, X., Zhang, M., Huang, B., Zheng, C., Zhang, P., Chen, W., Luo, Y., Chen, Y., Xia, H., et al. (2025). Enhancing diagnostic accuracy in rare and common fundus diseases with a knowledge-rich vision-language model. Nature Communications, 16(1). https://doi.org/10.1038/s41467-025-60577-9

Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce RetiZero, a vision-language model that incorporates knowledge from over 400 fundus diseases. The model is pre-trained on 341,896 fundus images with accompanying text descriptions gathered from diverse sources across multiple ethnicities and countries. RetiZero demonstrates exceptional performance across various downstream tasks including zero-shot disease recognition, image-to-image retrieval, clinical diagnosis assistance, few-shot fine-tuning, and cross-domain disease identification. In zero-shot scenarios, it achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases, while for image-to-image retrieval, it scores 0.950 and 0.886 respectively. Notably, RetiZero’s Top-3 zero-shot performance exceeds the average diagnostic accuracy of 19 ophthalmologists from Singapore, China, and the United States. The model particularly enhances clinicians’ ability to diagnose rare fundus conditions, highlighting its potential value for integration into clinical settings where diverse eye diseases are encountered.

Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

A*STAR Central Research Fund (“Robust and Trustworthy AI system for Multi-modality Healthcare”)

This research / project is supported by the A*STAR - Advanced Manufacturing and Engineering (AME) Programmatic Fund
Grant Reference no. : A20H4b0141

This research / project is supported by the National Medical Research Council - Clinician Scientist Award - Senior Investigator
Grant Reference no. : MOH-CSASI22jul-0001

https://oar.a-star.edu.sg/communities-collections/articles/23209

2041-1723

Institute of High Performance Computing 

https://doi.org/10.1038/s41467-025-60577-9