SuperPoint Outline

FRAMEWORK

MAINLY THREE PARTS（pic above）：

1. Interest Point Pre-Training

   1. use synthetic dataset（easy to get corner pts, e.g. L/Y/T junctions...）

   2. train base detector（what is the detector‘s arch?）

      

   3. transfer to real img（next steps）

2. Interest Point Self-Labeling（sample random homography，generate pseudo-GT）
   1. use base detector to inference（init interest pts in real img）
   2. use homographic adaption（detail？）
3. Joint Training（interest pts and descriptor）
   1. loss
   2. network arch

METHOD DETAIL

NETWORK DESIGN

1. encoder-decoder architecture

   1. shared encoder（advantages？）
   2. two heads

2. Shared Encoder

   1. VGG-styled

   2. pixel cells

      three 2×2 non-overlapping max pooling operations in the encoder result in 8 × 8 pixel cells

3. Interest Point Decoder

   1. NO upsampling layers（high computation & unwanted checkerboard artifacts）

   2. designed the interest point detection head（with an explicit decoder）

      This decoder has no parameters, and is known as “sub-pixel convolution” or “depth to space” in TensorFlow or “pixel shuffle” in PyTorch

4. Descriptor Decoder

   1. similar to UCN（Universal Correspondence Network）
   2. perform bicubic interpolation of the descriptor and then L2-normalizes（fixed）

LOSS FUNC

HOMOGRAPHY

1. formulation

   

   • $I$: input image
   • $x$: resulting interest points
   • $f_\theta$: network
   • $\mathcal{H}$: homography

2. improved super-point detector

   

3. choosing homographies

   

4. Iterative Homographic Adaptation

   

RESOURCE

rpautrat/SuperPoint: Efficient neural feature detector and descriptor (github.com)

magicleap/SuperPointPretrainedNetwork: PyTorch pre-trained model for real-time interest point detection, description, and sparse tracking (https://arxiv.org/abs/1712.07629) (github.com)

一种深度学习特征SuperPoint (qq.com)

腾讯优图荣获CVPR2021 Image Matching Workshop双赛道冠亚军 (qq.com)