Hu, W., Zhang, Y., Liang, Y., Yin, Y., Georgescu, A., Tran, A., Kruppa, H., Ng, S.-K., & Zimmermann, R. (2022). Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery. Proceedings of the 30th ACM International Conference on Multimedia. https://doi.org/10.1145/3503161.3548102
Abstract:
Street-view imagery provides us with novel experiences to explore different places remotely. Carefully calibrated street-view images (e.g., Google Street View) can be used for different downstream tasks, e.g., navigation, map features extraction. As personal high-quality cameras have become much more affordable and portable, an enormous amount of crowdsourced street-view images are uploaded to the internet, but commonly with missing or noisy sensor information. To prepare this hidden treasure for “ready-to-use”
status, determining missing location information and camera orientation angles are two equally important tasks. Recent methods have achieved high performance on geo-localization of street-view images by cross-view matching with a pool of geo-referenced satellite imagery. However, most of the existing works focus more on geo-localization than estimating the image orientation. In this work, we re-state the importance of finding fine-grained orientation for street-view images, formally define the problem and provide a set
of evaluation metrics to assess the quality of the orientation estimation. We propose two methods to improve the granularity of the orientation estimation, achieving 82.4% and 72.3% accuracy for images with estimated angle errors below 2 degrees for CVUSA and CVACT datasets, corresponding to 34.9% and 28.2% absolute improvement compared to previous works. Integrating fine-grained orientation estimation in training also improves the performance on geo-localization, giving top 1 recall 95.5%/85.5% and 86.8%/80.4%
for orientation known/unknown tests on the two datasets.
License type:
Attribution 4.0 International (CC BY 4.0)
Funding Info:
This research / project is supported by the Economic Development Board of Singapore - Industrial Postgraduate Program
Grant Reference no. : S18-1198-IPP-II