Kong, L., Xie, S., Hu, H., Ng, L. X., Cottereau, B., Ooi, W. T. (2023). RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems (Vol. 36, pp. 21298–21342). Curran Associates, Inc.
Abstract:
Depth estimation from monocular images plays an important role in real-world visual perception systems. Existing learning-based depth estimation models are trained and tested on meticulously cleaned data while ignoring out-of-distribution (OoD) situations. Common corruptions, however, tend to occur in practical scenarios, especially for safety-critical applications like autonomous driving. To fill in this gap, we present a comprehensive robustness test suite dubbed RoboDepth consisting of 18 corruptions from three categories: i) weather and lighting conditions; ii) sensor failure and movement; and iii) data processing issues. Then, we conduct a comprehensive benchmark on 42 existing depth estimation models from indoor and outdoor scenes, to evaluate their robustness under corruptions. Our benchmark results indicate that, due to the lack of a suitable robustness evaluation toolkit, state-of-the-art depth estimation models are at risk of being vulnerable to common corruptions. We further make in-depth discussions on the design considerations of building more robust depth estimation models, from aspects including pre-training, augmentation, modality, model capacity, and learning paradigm. We hope our benchmark can lay a solid foundation and facilitate robust OoD depth estimation.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation, Prime Minister Office - CREATE – Intelligent Modelling for Decision-Making in Critical Urban Systems (DesCartes) program
Grant Reference no. : NA