Matthew Strosnick – Project 1

1. My Feature Descriptor

I decided to make my custom descriptor an augmented version of the MOPS descriptor. For preprocessing, I convert the input image to grayscale and then create a derivative image in the X and Y directions using the Sobel operator. Then, I smooth the derivative images using a 7x7 Gaussian kernel with a sigma of approximately 2.0. When computing the descriptor I evenly sample 25 pixels from a 40x40 region, which is oriented by the feature’s angle and centered on the feature’s location. By rotating the region of interest some rotation invariance is provided. Next, at each sample point I add three elements to the descriptor vector: intensity, x derivative, and y derivative. Once the descriptor is fully computed, I normalize all of the intensity elements, and then normalize all of the derivative elements. This normalization provides some illumination invariance.

2. Major Design Choices

The first major design choice I made was to orient my key points using arctan(dy/dx), where dy and dx are Sobel responses at the location of a Harris key point. My next major design choice was to normalize the MOPS descriptor. This lowered the quality of the results in some cases, but overall provides more robustness. Another major design choice was to implement Adaptive non-maximum suppression. This lowered the quality of the results by a little bit, but greatly increased matching efficiency.

3a. Performance - ROC Curves

ROC curves were made using the test images Graf and Yosemite in order to visualize the quality of the algorithms. Additionally, threshold curves were made for the same images to indicate potentially good threshold values for each algorithm. Results can be seen below in Figures 1 through 4.

Figure 1: Graf ROC Figure 2: Yosemite ROC

Figure 3: Graf Threshold Figure 4: Yosemite Threshold

3b. Performance – Harris Images

Two Harris response images, enhanced via Adobe Photoshop, can be seen below in Figures 5 and 6.

Figure 5: Graf Harris Responses Figure 6: Yosemite Harris Responses

3c. Performance – Average AUC’s

Using the benchmark feature and four image sets, average AUC values were computed for each algorithm. Results can be seen in the table below, the best AUC value for each image set is highlighted.

Algorithm	Graf AUC	Leuven AUC	Bikes AUC	Wall AUC
Simple+SSD	0.559169	0.267742	0.422516	0.528014
MOPS+SSD	0.498034	0.512514	0.589116	0.527970
Custom+SSD	0.636777	0.617762	0.709942	0.591513
Simple+Ratio	0.531024	0.550753	0.492657	0.571818
MOPS+Ratio	0.553496	0.618500	0.629442	0.592248
Custom+Ratio	0.604234	0.631448	0.659784	0.588806

4. Strengths and Weaknesses

Strengths:

· Harris key points are relatively stable when faced with translation, rotation, and affine changes in illumination

· The MOPS and Custom descriptors are normalized, providing illumination invariance

· Adaptive non-maximum suppression provides a bounded matching efficiency

Weaknesses:

· The descriptors are not scale invariant

· Adaptive non-maximum suppression seemed to generally lower average AUC outcomes

· Matching could be implemented more efficiently

5. My Own Images

In order to further test the results, I took two of my own images. These two images are of the inside of a car at night and introduce a reasonable amount of complexity. On the left I selected several features, and on the right are the matches (Figure 7). The features and matching do well along the edge of the steering wheel and near the radio. The features and matching perform poorly in the smooth dark regions due to ambiguity in those regions.

Figure 7: Selected features (left) and corresponding matches (right)

6. Extra Credit

For extra credit I implemented:

· Adaptive Non-Maximum Suppression

· Normalized MOPS

Additionally, I tried to use Correlation rather than SSD for matching, but this did not work better than the ratio test so I commented it out in my code.