Federated benchmarking of medical artificial intelligence with MedPerf

Page view(s)
14
Checked on Jul 05, 2024
Federated benchmarking of medical artificial intelligence with MedPerf
Title:
Federated benchmarking of medical artificial intelligence with MedPerf
Journal Title:
Nature Machine Intelligence
Keywords:
Publication Date:
17 July 2023
Citation:
Karargyris, A., Umeton, R., Sheller, M. J., Aristizabal, A., George, J., Wuest, A., Pati, S., Kassem, H., Zenk, M., Baid, U., Narayana Moorthy, P., Chowdhury, A., Guo, J., Nalawade, S., Rosenthal, J., Kanter, D., Xenochristou, M., Beutel, D. J., … Chung, V. (2023). Federated benchmarking of medical artificial intelligence with MedPerf. Nature Machine Intelligence, 5(7), 799–810. https://doi.org/10.1038/s42256-023-00652-2
Abstract:
AbstractMedical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform.
License type:
Attribution 4.0 International (CC BY 4.0)
Funding Info:
This research / project is supported by the A*STAR - Central Research Fund
Grant Reference no. : NA

This research / project is supported by the A*STAR - Career Development Fund
Grant Reference no. : C222812010

This research / project is supported by the National Research Foundation - AI Singapore Programme
Grant Reference no. : AISG2-TC-2021-003
Description:
ISSN:
2522-5839
Files uploaded:

File Size Format Action
2023-nmi-federated-benchmarking.pdf 1.67 MB PDF Open