How does Lowe compute the "repeatability" of his SIFT Algorithm?

How does Lowe compute the “repeatability” of his SIFT Algorithm?

• In his paper of 2004, "Distinctive Image Features from Scale-Invariant Keypoints", he gave many figures of "repeatability" as a function of XXX, for example, figure 3,4 and 6, but he did not elaborate how to compute the "repeatability". He actually gave an simple explanation of "repeatability" in figure 3 of page 8, which is "the percent of keypoints that are repeatably detected at the same location and scale in a transformed image". However, 1) how can we know weather a keypoint is detected in the transformed image, simply by checking the keypoints one by one? I do not think it is practical, since there are thousands of keypints. 2) How close can we consider a keypoint being repeatably detected in the SAME LOCATION? 3 pixels, 6 pixels? Or not by pixels at all? I do not know how can I upload the paper "Distinctive Image Features from Scale-Invariant Keypoints". Here is the link: http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf

• Answer:

  If you notice in the first paragraph on page 9, the author goes into a bit more detail on the tolerance for saying the keypoint is detected in the same location. First, "the same scale" is given as being within a factor of sqrt(2) of the correct scale, which must either be known to the operator or hand labeled in the data set. Secondly, "the same location" is defined as being within σ pixels in the x and y directions, where "σ is the scale of the keypoint (defined from equation (1) as the standard deviation of the smallest Gaussian used in the difference-of-Gaussian function)" As to your more general question, the answer is unfortunately that one must indeed check the keypoints exhaustively. You gather up all the location/scale pairs where your detector fires on the transformed image, and you compare these to the true locations back in the original image. You need to have hand-recorded keypoint locations, or run a simple algorithm first (like using the basic KLT keypoint detector and just storing a list of the coordinates of the keypoints it finds and pretendting like these are "ground truth"... you'll suffer some accuracy this way but you'll be more able to automate the process). Large scale grunt work like this (writing code to exhaustively check a ton of keypoints) is usually the stuff that gets farmed out to grad students. Blerg.