Data-driven 3D Local Features for Point Cloud Registration

Developing robust and accurate methods for aligning 3D point clouds by learning discriminative feature descriptors is a long standing endeavor. Point clouds, which represent the shape and geometry of objects in 3D space, are critical for applications like 3D reconstruction, autonomous navigation, and augmented reality. Traditional methods rely on handcrafted features, but data-driven approaches leverage machine learning, particularly deep learning, to extract local features from point clouds. These learned features capture complex patterns and are invariant to transformations such as rotation and scale, enabling more precise and efficient point cloud registration in challenging environments.

Through a series of publications, we have explored the development of 3D local features suitable for use in autonomous driving as well as robotics applications like bin picking and navigation. Our methods can also be used for reconstructing scenes by aligning information from different views or scans.

2022

  1. zhao20223dpointcaps.jpg
    3dpointcaps++: Learning 3d representations with capsule networks
    Yongheng Zhao, Guangchi Fang, Yulan Guo, and 3 more authors
    Int. Journal of Computer Vision (IJCV), 2022
    arXiv HTML Code Website

2022

  1. huang2022multiway.jpg
    Multiway non-rigid point cloud registration via learned functional map synchronization
    Jiahui Huang, Tolga Birdal, Zan Gojcic, and 2 more authors
    IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2022
    arXiv Code Website

2021

  1. huang2021multibodysync.jpg
    Multibodysync: Multi-body segmentation and motion estimation via 3d scan synchronization
    Jiahui Huang, He Wang, Tolga Birdal, and 4 more authors
    In IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2021
    arXiv Video Code

2020

  1. gojcic2020learning.jpg
    Learning multiview 3d point cloud registration
    Zan Gojcic, Caifa Zhou, Jan D Wegner, and 2 more authors
    In IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2020
    arXiv Code

2019

  1. zhao20193d.jpg
    3D point capsule networks
    Yongheng Zhao, Tolga Birdal, Haowen Deng, and 1 more author
    In IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2019
    arXiv Code Website

2019

  1. deng20193d.jpg
    3D Local Features for Direct Pairwise Registration
    Haowen Deng, Tolga Birdal, and Slobodan Ilic
    In IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2019
    arXiv

2018

  1. deng2018ppf.JPG
    PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors
    Haowen Deng, Tolga Birdal, and Slobodan Ilic
    In Eur. Conf. Computer Vision (ECCV), 2018
    arXiv

2018

  1. deng2018ppfnet.JPG
    Ppfnet: Global context aware local features for robust 3d point matching
    Haowen Deng, Tolga Birdal, and Slobodan Ilic
    In IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2018
    arXiv