MAC-VO: Metrics-Aware Covariance for Learning-based Stereo Visual Odometry
Carnegie Mellon University
* Equal Contribution.
Abstract
We propose MAC-VO, a novel learning-based stereo VO that leverages the learned metrics-aware matching uncertainty for dual purposes: selecting keypoint and weighing the residual in pose graph optimization. Compared to traditional geometric methods prioritizing texture-affluent features like edges, our keypoint selector employs the learned uncertainty to filter out the low-quality features based on global inconsistency. In contrast to the learning-based algorithms that rely on the scale-agnostic weight matrix, we design a metrics-aware spatial covariance model to capture the spatial information during keypoint registration. Integrating this covariance model into pose graph optimization enhances the robustness and reliability of pose estimation, particularly in challenging environments with varying illumination, feature density, and motion patterns. On public benchmark datasets, MAC-VO outperforms existing VO algorithms, even some SLAM algorithms in challenging environments. The covariance-aware framework also provides valuable information about the reliability of the estimated poses, which can benefit decision-making for autonomous systems.
Video
Results
Interactive 3D Demo, drag for viewport rotation and scroll to zoom in/out
Loading a demo may take several seconds, depending on your network condition.
More Interactive 3D Demos
We present some additional maps and trajectories reconstructed by the MAC-VO using default configuration (without fine-tuning). Click to open interactive 3D demo.