Single-Domain Generalization via Path Flatness-Aware Optimization of Loss Landscapes

Single-Domain Generalization via Path Flatness-Aware Optimization of Loss Landscapes

IEEE Transactions on Neural Networks and Learning Systems

10.1109/TNNLS.2025.3616593

https://doi.org/10.1109/tnnls.2025.3616593

Zizhou Wang, Yan Wang, Yangqin Feng, Jiawei Du, Joey Tianyi Zhou, Rick Siow Mong Goh, Yong Liu, Liangli Zhen

01 March 2026

Wang Z, Wang Y, Feng Y, Du J, Zhou JT, Goh RS, Liu Y, Zhen L. Single-Domain Generalization via Path Flatness-Aware Optimization of Loss Landscapes. IEEE Transactions on Neural Networks and Learning Systems. 2026.

Domain generalization (DG) methods traditionally rely on multiple source domains to achieve the robust performance across unseen target domains. However, single-DG (SDG) presents a more practical paradigm by learning from a single source domain, addressing scenarios where access to multiple domains is limited. While existing SDG approaches primarily focus on data augmentation and style transfer techniques to enhance the model robustness, these methods often incur substantial computational overhead and may inadequately capture the complexity of real-world domain shifts. In this article, we propose path flatness-aware optimization (PFO), an optimization framework that addresses the fundamental challenges of SDG. Unlike conventional approaches that rely on the synthetic data generation, PFO identifies and exploits regions of flat minima within the optimization landscape of deep neural networks. The framework employs an iterative optimization strategy to construct a path through the parameter space along which an ensemble of candidate models achieves the minimal empirical risk. The initialization of this optimization path is achieved through the strategic interconnection of model instances, each originating from carefully selected anchor points that are computationally determined through the systematic analysis of classification decision manifolds. This optimization path serves as a mechanism for implicit distribution alignment between source and target domains within the loss landscape, consequently enhancing the model’s capacity for cross-DG. Empirical evaluation on multiple benchmark datasets demonstrates significant performance improvements in cross-DG, validating the efficacy of our approach.

Publisher Copyright

This research / project is supported by the National Research Foundation of Singapore - AI Singapore
Grant Reference no. : AISG2-TC-2021-003

This research / project is supported by the National Research Foundation of Singapore - AI Singapore
Grant Reference no. : AISG4-GC-2023-007-1B

This research / project is supported by the Singapore Digital Trust Centre - Innovation Grant Award
Grant Reference no. : DTC-IGC-02

© 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/23378

2162-237X
2162-2388

Institute of High Performance Computing