In developing the Panorama code most of the design was fairly straightforward and didn't require many true design choices. The spherical warping was based on the formulas given in lecture. RANSAC feature matching by counting inliers and doing a least squares fit also followed the standard algorithm. When actually running the code we were able to tune some of the algorithm parameters; for the test pictures using the default 1000 iterations and 4.0 size inlier threshold. For pictures that we shot ourself we found a tighter inlier threshold of 3.0 pixels worked better. When performing blend images we initially had it working well for homography image transformations but had dark sections appear when blending images that had been pre-warped because the black sections would show up in the final artwork. To solve this we simply checked to see if pixels were completely black before adding them to the panorama. Only pixels that were not completely black in color would be added which proved to be a simple method to fix this problem. The only minor problem is that some images when warped have pixels that are darker along the edge of the warping. This causes them to have a darker color that is not black. These pixels cause barely visible single pixel lines to appear between along image edges. One could fix this by requiring neighbor pixels to not be black, but we thought the existing version looked good. Also 100 pixels worked well for the blend width. Shortening the blend width to less than 100pixels removed some radial distortion in the handheld images but also resulted in more jagged transitions between objects that were distorted across image boundaries.
Also we never did any extra credit.
Soultion Executable:
I don't remember if our solution executable was in the final code zip file that we uploaded so we included a copy of it with the artifact submission in case it wasn't included. Executable
Test Images (All Images are clickable links to their full size images.)
Yosemite: Using SIFT Features with pre-warping
Campus 360 degrees: Using SIFT Features with pre-warping
360 degree Applet Viewer of Campus Image:
Hand-held Panoramas:
Pictures were shot with Peter's Canon SD870 IS handheld camera this past weekend. Focal length parameter was calculated using the EXIF data method. No parameters were calculated for radial distortion and as a result there is some distortion visible in the panoramas but it still looks pretty good and generally the program was able to match images well. The calculation was (image width)*(focal length mm)/(sensor width mm) = 557 pixels for Peter's camera.
The first is a 180 degrees set of pictures from Yankees Stadium Oct 13 when the Tigers were playing the Yankees: Using SIFT Features with pre-warping
Homography:
Additionally we examined the performance of just using homography transformations without pre-warping the images and found as we expected that images on the side start to get incredibly distorted and skewed as the homography tries to correct for the perspective distortion. Below we show how this breaks down as you start to add more pictures. As the total angle increases the homography starts to break down more and more. When you run this on a full 360 image set the program just crashes as it can't find an appropriate transformation.
Yankees: Combine 2 pictures with only Homography transformation, SIFT features:
Yankees: Combine 3 pictures with only Homography transformation, SIFT features:
Yankees: Combine 4 pictures with only Homography transformation, SIFT features:
Yankees: Combine 5 pictures with only Homography transformation, SIFT features:
Also interesting is that the program keeps the 1st image square so if you reverse the order of the images you get the opposite direction distortion.
Yankees Reverse order: Combine 5 pictures with only Homography transformation:
We also tried using MOPs descriptors instead of SIFT features with the pre-warped image translation and saw much worse performance. MOPs couldn't overcome the radial distortion in order to properly align the images.
Yankees: 5 pictures MOPs descriptors, pre-warp
For our 360 degree handheld Peter shot pictures in Old Westbury Gardens on Long Island on Oct 22. One interesting thing to note is that the green arch strutures was curved in a semicircle, but in the panorama it appears flat as it is at a constant radius from where the images were shot at.
Gardens: SIFT Features with pre-warping
Thanks for visiting!