CS 4670 Introduction to Computer Vision (Fall 2010)

Project 1: Image Filtering

Released: Friday, Sep. 3, 2010
Part A Due: Monday, Sep. 13, 2010
Part B Due: Friday, Sep. 17, 2010
Last Modified: Tuesday, Sep. 14, 2010

Quick Link: Overview Description Getting Started Code to Write Submission

Overview

Image filtering allows you to apply various effects on photos. In this project, you will implement image filters in C/C++ on your computer and Nokia N900 camera phone. You will get to experiment with image filtering and build up some programming experience in OpenCV and FCam API, which will help you with later projects.

Description

Recall that a filter can be represented as a (2k + 1) × (2k + 1) matrix. You will implement a function for convolving an image with several filters as follows.

The size of Gaussian Filer should be 5 × 5 and the size of Sharpen Filter should be 3 × 3. For Sobel Filter, you need to implement both x and y direction. For Median Filter, the k is not fixed but can be passed when running the program.

Getting Started

To begin with, you need to set up enviroment for OpenCV and FCam on your computer. Take a look at OpenCV installation guide before getting started. An instruction for setting up FCam can be found here. If you are using Ubuntu, make sure the following packages are installed by typing the following.

sudo apt-get install libcv-dev libcvaux-dev libhighgui-dev build-essential
If running under Windows, you will need to install OpenCV v2.1 yourself from here.

Next download skeleton code for project 1. Unpack the code and you will find README.txt and read it carefully. You can use make_pc to compile the program and make_pc clean to clean. To run the program, goto project directory and type ./cs4670 -h in your terminal and it should print out the usage of the program. The main (and the only) source file you will need to modify is Project1.cpp. You shouldn't change the interface since the same interface will be called for the desktop program and cell phone program as well.

The main class for holding an image in OpenCV is IplImage. Here're the members that you might need for IplImage. 

		IplImage
  		|-- int  width;         	// image width in pixels
  		|-- int  height;        	// image height in pixels
You may also use CvMat, which is the class representing a matrix in OpenCV, to represent the kernel. The member you need to care is as follows. 
		CvMat                      	// 2D array
  		|-- int   rows, cols;    	// dimensions
In general, you can think of a gray image as an m-by-n matrix. An RGB image can be thought of as three sperate images, one for each of three color channels: red, green, and blue (a fourth channel, the alpha channel will be unused in this assignment but will be used in later projects). For the functions that involve color image processing, you will simple split the color image into the three channels, operate on each channel as a grayscale image, then put the channels back together into a single color image.

Here're a few additional links if you want to learn more of IplImage and CvMat.

Apart from these two classes, there are several useful macros to access the data in IplImage and CvMat. Examples of these macros can be found here. Here's an example of accessing the element of CvMat. Note the *mat here, which is different from the examples you might find at the previous link.
float a32 = CV_MAT_ELEM(*mat, float, 3, 2);

We provide a test image for project 1. You can also find this image in the project package. You should use this test image for Gaussian Filter and you should use your own test images for the others..

Code to Write

The only file you need to edit is the Project1.cpp. Here's a list of functions you need to implement.

Note that the int argument in the last five functions is unused except median. Your solution for the other four shouldn't depend on such argument.

  1. Implement the filter2D function
    2. This is a generic filter function that takes an image and a kernel, produces the filtered image. You will need to describe how you handle the boundary cases in the writeup.
  2. Implement the RGB2Gray function
    3. The function should convert an RGB image A (an m-by-n-by-3 matrix) to a grayscale image B (an m-by-n matrix), which should be used for Sobel Filter. The conversion is a linear combination of the R, G and B component. Note that conversion from RGB to grayscale is not unique, as there are many possible weightings of the three color channels. Tell us what the exact parameters you use and include them in the submission.
  3. Implement the gaussian function
    4. The function should first split an RGB image into seperate images per color channel, which can be done by calling the built-in OpenCV function cvSplit. Then you will create a Gaussian kernel and call the generic filter function filter2D on each channel. At the end, you need to merge back those seperate images into an RGB image by calling cvMerge. Note that cvSplit splits an image into four seperate images. The fourth channel argument should be NULL. Similarly, the fourth channel argument of cvMerge should be NULL as well. The filter should be 5 × 5. In the submission,you need to specify how to handle the image values outside of the image A. In addition, you cannot use any other built-in functions in OpenCV doing the filtering. Remember to implement convolution and not cross-correlation. Here's how you will get when applying the Gaussian Filter.
  4. Implement the sharpen function
    5. Similar to the gaussian. The function works on RGB image and you should frist split an RGB image, run sharpen filtering on each channel and merge them back into an RGB image. The size of the filter is 3 × 3.
  5. Implement the sobel_x and sobel_y functions
    6. The function should run sobel_x and sobel_y filtering on grayscale image A. So you need to call RGB2Gray first before applying the generic filter function.
  6. Implement the median function
    7. The function should run median filtering on RGB image A using a (2 × k + 1)-by-(2 × k + 1) window. Similar to Gaussian and Sharpen Filter, you begin with splitting the RGB image into three channels. On each channel you run median filter and merge three channels back.

Submission

Part A: Desktop Program (Due: Friday, Sep. 10, 2010 )

You need to create a webpage showing your images before and after applying different filters. The webpage should contain several sections. The first section should tell us how you choose the weights for RGB to gray conversion. Next section should explain how you handle the boundary case in your generic filter function filter2D. Then, each section corresponds to a filter you are supposed to implement. For Gaussian Filter, show the result using our given image. For the others, you need to include your test images and show the results of them. You need to explain the filter weights for the Gaussian and Sharpen filter, and how you come up with those weights, in the corresponding section.

Compress the webpage into a zip file called webpage.zip and compress your source codes into code.zip and submit them to the CMS

Part B: Cell Phone Program

You'll first need to sign up for picking up the phone since we don't have enough phones, which should be done before Tuesday Sep 14. There's a signup sheet in front of Upson 4144. There're several time slots for picking up the phone, i.e. Tuesday Wednesday Thursday 4:00pm - 4:30pm. You will have to return the phone and give your TA a demo show by next day from 11:00am - 12:00pm.

When you signup, make sure there are phones available for that day and you should put your netid in the corresponding cell. For example, you want to pick the phone "alf" at Tuesday 4:00pm - 4:30pm. You can put your netid in that cell. Each phone can only be owned by one student per time slot. So do not write down your netid if you find that cell already has filled up.

Then at the time you signed up for, go to Upson 4144 to pick up the phone. You'll have one night testing your code. You should also give a demo show in the lab (Upson 317) and return the phone there.

Instructions for building part B can be found in the README within the project tarball on this site. Note, the program will display a live camera viewfinder, a sliderbar, and six buttons. The buttons 1-5 will change the selected filter. The x button will close the program. The sliderbar changes the parameter k for the median filter. Pressing the camera button half-way on the top left of the phone will make the image auto-focus and pressing the button fully will take a picture, apply a filter, and save it to the flash memory. From there, you can close the program, open the applications menu, and select 'Photos.' You should find your filtered photo in there.