L2G-NeRF: Local-to-Global Registration for Bundle-Adjusting

Neural Radiance Fields

CVPR 2023

Yue Chen1*, Xingyu Chen1*, Xuan Wang2†, Qi Zhang3, Yu Guo1†, Ying Shan3, Fei Wang1

*Equal Contribution    Corresponding Author    1Xi'an Jiaotong University    2Ant Group    3Tencent AI Lab


Neural Radiance Fields (NeRF) have achieved photorealistic novel views synthesis; however, the requirement of accurate camera poses limits its application. Despite analysis-by-synthesis extensions for jointly learning neural 3D representations and registering camera frames exist, they are susceptible to suboptimal solutions if poorly initialized. We propose L2G-NeRF, a Local-to-Global registration method for bundle-adjusting Neural Radiance Fields: first, a pixel-wise flexible alignment, followed by a framewise constrained parametric alignment. Pixel-wise local alignment is learned in an unsupervised way via a deep network which optimizes photometric reconstruction errors. Frame-wise global alignment is performed using differentiable parameter estimation solvers on the pixel-wise correspondences to find a global transformation. Experiments on synthetic and real-world data show that our method outperforms the current state-of-the-art in terms of high-fidelity reconstruction and resolving large camera pose misalignment. Our module is an easy-to-use plugin that can be applied to NeRF variants and other neural field applications.

Overview video

Neural Image Alignment (2D)

Rigid Alignment

Homography Alignment

NeRF (3D): Synthetic Objects


Ablation study

NeRF (3D): Real-World Scenes


Ablation study

More Examples


  title={Local-to-global registration for bundle-adjusting neural radiance fields},
  author={Chen, Yue and Chen, Xingyu and Wang, Xuan and Zhang, Qi and Guo, Yu and Shan, Ying and Wang, Fei},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},