A Deep Network to Find Correspondences from Multiple Objects
This is a project I did with a graduate student, Youye Xie. We addressed the problem of finding correct 3D to 2D point correspondences, from a set of putative correspondences. Namely, we want to find the inlier matches and classify those inliers into multiple objects.
This problem comes up in object recognition. Say that you have a color and depth training image of some objects. You then have a test image, and you want to find occurrences of those objects in the test image. A classic way to do this is to find features in both the template and test images, and then match them. Of course, a lot of those feature matches are incorrect. What we want to do is to find the inlier matches, and group those into distinct objects.
You can solve this problem with RANSAC, but RANSAC is an iterative algorithm and is very slow, especially when there are a lot of outlier (incorrect) matches. Instead, we developed a deep network that identifies inlier matches coming from multiple objects, using only one pass through the network. As a result, our method is much faster than competing methods.
Our method finds the inlier matches between the template image (top) and the test image (bottom), and classifies them as coming from two different objects.
This work is described in detail in our paper: Y. Xie, Y. Tang, G. Tang, and W. Hoff, "Learning To Find Good Correspondences Of Multiple Objects." International Conference on Pattern Recognition (ICPR), 2021. (pdf)
The code is available on Github at this link: https://github.com/youyexie/Learning-To-Find-Good-Correspondences-Of-Multiple-Objects.
See our slides for our presentation, and the video of the presentation below.