Liu, Z., Liu, J., Chen, W., Wu, X., & Li, Z. (2022). FAMINet: Learning Real-Time Semisupervised Video Object Segmentation With Steepest Optimized Optical Flow. IEEE Transactions on Instrumentation and Measurement, 71, 1–16. https://doi.org/10.1109/tim.2021.3133003
Abstract:
Semi-supervised video object segmentation (VOS) aims at segmenting a few moving objects in a video sequence, where these objects are specified by the first frame’s annotation. The optical flow has been considered in many existing semisupervised VOS methods to improve the segmentation accuracy.
However, due to high complexity of optical flow estimation, the optical flow-based semi-supervised VOS methods are unable to run in real time. To address this problem, a FAMINet which consists of a feature extraction network (F), an appearance network (A), a motion network (M), and an integration network
(I) is proposed in this paper. The appearance network outputs an initial segmentation result based on objects’ static appearances. The motion network estimates the optical flow via very few parameters, which are optimized rapidly by a relaxed steepest descent algorithm online. Finally, the integration network refines
the initial segmentation result using the optical flow. Extensive experiments demonstrate that the FAMINet outperforms other state-of-the-art semi-supervised VOS methods on the DAVIS and YouTube-VOS benchmarks, and achieves a good trade-off between accuracy and efficiency.
License type:
Publisher Copyright
Funding Info:
This work is supported by National Natural Science Foundation of China
under Grant No. 61620106012 and No. 61573048.