Montreuil, Y., Heng, Y.S., Carlier, A., Ng, L.X. ; Ooi, W.T.. (2025). A Two-Stage Learning-to-Defer Approach for Multi-Task Learning. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research, 267:44726-44749 Available from https://proceedings.mlr.press/v267/montreuil25b.html.
Abstract:
The Two-Stage Learning-to-Defer (L2D) framework has been extensively studied for classification and, more recently, regression tasks. However, many real-world applications require solving both tasks jointly in a multi-task setting. We introduce a novel Two-Stage L2D framework for multi-task learning that integrates classification and regression through a unified deferral mechanism. Our method leverages a two-stage surrogate loss family, which we prove to be both Bayes-consistent and (G,R)-consistent, ensuring convergence to the Bayes-optimal rejector. We derive explicit consistency bounds tied to the cross-entropy surrogate and the L1-norm of agent-specific costs, and extend minimizability gap analysis to the multi-expert two-stage regime. We also make explicit how shared representation learning—commonly used in multi-task models—affects these consistency guarantees. Experiments on object detection and electronic health record analysis demonstrate the effectiveness of our approach and highlight the limitations of existing L2D methods in multi-task scenarios.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation, Prime Minister Office, CREATE - DesCartes under Campus for Research Excellence and Technological Enterprise (CREATE) program - DesCartes
Grant Reference no. : NA
This research / project is supported by the National Research Foundation, Singapore - AI Singapore Programme
Grant Reference no. : AISG2-PhD-2023-01-041-J
Description:
Proceedings of the 42nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s)