CS4670 Project 2

Jeremy Feinstein & Andrew Imm

Major Design Decisions

Feature Detection

In order to compute the Harris features, we first convolve the image with 5x5 Sobel filters for the x and y partial derivatives. We chose to use 5x5 kernels because we thought using a better approximation of the x and y partials would lead to better results and would avoid ambiguity more than the 3x3 Sobel filters would. Then, we compute the elements of the Harris matrix using the results from the Sobel convolutions, weighting each pixel in the 5x5 window with its corresponding pixel in the 5x5 Gaussian mask. Again, we chose to use the 5x5 Gaussian mask so that our features would be less ambiguous and would lead to less false matches.

We use the equation listed on the project site to calculate c, and then calculate the angle associated with each window as arctan(Iy/Ix). This angle corresponds to the angle of maximum increase for the center pixel of the window. Instead of calculating the eigenvalues and eigenvectors for each window (which would be computationally expensive), we instead chose to use this angle to be our unique feature angle. This angle is also invariant to rotation, which is a necessity for feature detection.

We also follow the instructions on the project site and scan through a 5x5 neighborhood around each feature to determine whether or not it is a local maximum and above the threshold we set (0.1). If a feature meets those two qualifications then it is marked as a true feature.

Descriptors

Computing the MOPS descriptors for each Harris required some very involved image manipulations. We iterate over each of the marked features and rotate them so that their characteristic angle is aligned with the positive x axis. This involves several distinct steps. First, we copy a 64x64 pixel window around the feature into a new image. We determined that it would have to be 64x64 so that we can later pull out a proper 41x41 pixel window around the feature's center. We then apply 3 transformations to this matrix: a translation to move the center to the origin, a rotation of the feature's characteristic angle, and the inverse translation to move the center back to its original position. From there, we apply a Gaussian blur to the 64x64 pixel window and then subsample a 41x41 pixel window around the center of the feature to get our 8x8 oriented patch. Doing all these steps on a new image (the 64x64 pixel window) saves us a lot of time as opposed to applying the transformations to the entire original image.

The remainder of the calculations for the MOPS descriptors are as described on the project site, where we normalize the pixel values based on the mean and standard deviation within the window.

Feature Matching

No significant design decisions were made with respect to applying the ratio test. Our code takes a form similar to that of the supplied function ssdMatchFeatures.

Performance

Benchmarks:

Bikes Average AUC: 0.611589

Leuven Average AUC: 0.614889

Wall Average AUC: 0.600742