Geometric primitives such as points and lines extracted from digital images are inherently uncertain. Although camera pose estimation from points or lines is a well-studied problem in computer vision, a systematic treatment of these uncertainties remains an open question for accurate and robust pose estimation. In this paper, we address this question by utilizing the uncertainty of points and lines to achieve robust and accurate pose estimation. We propose an accurate subset selection scheme of points and lines based on their uncertainties for pose estimation. First, the uncertainties of straight line and line segments under Hough
coordinate are introduced. Based on the uncertain points and lines, we derive the uncertainties of directional and distance constraints for pose estimation. We select these constraints with the lowest variance and apply them under direct linear transformation (DLT) for pose estimation. In experiment, the proposed method is evaluated against the other DLT-based methods and the state-of-arts in both synthetic and real image
datasets.