CS 5413: High Performance Systems and Networking (Fall 2014)
Lab3 - Packet Filter / Sniffer Framework

Introduction

In the lab 3, you will learn about kernel module programming and linux kernel network stacks. We strongly suggest you to read Chapter 2~5, 7 (about work queue) and 17 (about socket buffer) from Linux Device Drivers book and Linux netfilter Hacking HOWTO before you start this lab. (Note: the linked netfilter document is outdated and NF_IP_XXX must be replaced by NF_INET_XXX)

Your task is to write a kernel module that 1) allows you to open / block TCP connections (similar to IP table), 2) captures packets of your interest and delivers them to userspace (similar to pcap), and 3) performs deep packet inspection (DPI) on packets of your interest. There are many ways to hijack packets from linux kernel stack, and we will use netfilter for our purpose. Then, through character device interface and system calls such as open / close / read / ioctl, you will control the ip table and communicate with kernel space.

The assignment

You can load the module at the command line as follows:

 # insmod ./sniffer_mod.ko

Similarly you can unload the module as follows:

 # rmmod sniffer_mod

Then, you can control the kernel module invoking sniffer_control:

 # ./sniffer_control [mode] [src_ip] [src_port] [dst_ip] [dst_port] [action]

and read captured packets from kernel space by invoking sniffer_read:

 # ./sniffer_read [-i input file] [-o output file]

For example, to enable access to port 4000 on your local machine while capturing packets destined to the port, run

 # ./sniffer_control --mode enable --dst_ip localhost --dst_port 4000 --action capture

Then, you can display captured packets to your terminal screen as follows:

 # ./sniffer_read 
192.168.1.1:5000 -> 192.168.3.1:5000
......

Fetching and building the source

Start by downloading the skeletal code from CMS (sniffer.tar.gz), then copy it (using scp/rsync) to your home directory on one of your Fractus Cloud instances (recall that prompt> denotes your own machine, while # denotes the Fractus Cloud instance). Use the same image you created during Lab 0b. You should be able to get the files from your local box to your instance and build like this:

prompt>scp -i ~/.euca/id-rsa-kp-kl568-test sniffer.tar.gz root@128.84.9.XXX:~/
prompt>ssh -i ~/.euca/id-rsa-kp-kl568-test root@128.84.9.XXX
#  tar -xzf sniffer.tar.gz 
#  cd sniffer 
#  make

make will generate a kernel module binary file, namely sniffer_mod.ko, as well as two userspace applications: sniffer_control and sniffer_read. You can load the compiled module like this:

#  insmod ./sniffer_mod.ko

If the module is successfully loaded, you can see the module from the loaded module list:

#  lsmod | grep sniffer 
sniffer      12753 0

#  dmesg | tail -n 1 
[  XXX.XXXXXX] sniffer_init

dmesg prints the kernel debug buffer on the screen. What you print in kernel space using printk will be shown with dmesg. The combination of dmesg and printk is the easiest (but not the most efficient) way to debug the kernel module (LDD Chapter 4).
The provided kernel module by default creates a character device for you. You need to find out what the major number of the character device is (LDD Chapter 3).

#  cat /proc/devices | grep sniffer 
251 sniffer

Then, you need to create a device file to read from the character device interface.

#  sudo mknod sniffer.dev c 251 0

Now, you can read from the kernel module through the device file, for example,

# ./sniffer_read -i sniffer.dev

Above command will show nothing, however, because the kernel module and sniffer_read are not finished yet.

Specification

The module and sniffer_control and sniffer_read must behave as follows:

Filtering By default, the module drops all TCP packets except packets destined to port 22 of your local host. The module enables or disables TCP connections of four tuples (src_ip, src_port, dst_ip, dst_port) (or a flow) using ioctl calls from sniffer_control. Each member of four tuples can be "any", meaning any IP or port number is acceptable. Duplicated flows with different commands/actions will be overwritten, and handling conflicting rules is up to your implementation. For example, you can simply pick the oldest rule. You should mainly modify sniffer_nf_hook , and sniffer_ioctl.
Capturing When the action parameter is given as "capture", the module will capture and store packets in a buffer, which can be read from sniffer_read through read over the character device interface. read will be blocked when there is no packet in the buffer (use wait queue ). sniffer_read dumps packets following a format specified in sniffer_read.c to a file or stdout. Be careful of race condition while handling read, i.e. packets can be inserted into the buffer while sniffer_read is reading from the buffer. Also make sure at most one process can read from the dev file anytime (Use atomic counter).
DPI When the action parameter is given as "dpi", the module will perform deep packet inspection on every packet of the flow. For simplicity, the module will inspect the packet looking for a single signature. Also for simplicity, the signature can be hard-coded in the module. When a signature is detected, the module drops the packet and blocks further packets of the same flow.

Please finish Filtering part by Oct. 10th and check with one of TAs during the lab session.

Extra Credit

Implement a proc interface where you can list current ip table rules. For example:

# cat /proc/sniffer 
#   [command] [src_ip]       [src_port]  [dst_ip]       [dst_port] [action]
1    enable     any             any      127.0.0.1         4000      None
2    enable   128.84.154.162     80         any            any        DPI
...

Testing

You can test the kernel module by running your proxy from Lab 1 (or Lab 2). If you want the proxy to redirect any connections to port 4000 on your local machine to fireless.cs.cornell.edu:80 , then run

# ./sniffer_control --mode enable --dst_ip localhost --dst_port 4000
# ./sniffer_control --mode enable --src_ip fireless.cs.cornell.edu --src_port 80 
# cd <directory to your proxy> 
# ./tcp-proxy fireless.cs.cornell.edu 80 4000

The proxy should run without any problems.

How/What to hand in

You should submit two things:

A complete software distribution of your kernel module
sniffer_control, sniffer_read

You should also build the software distribution with the make dist command. To turn in your distribution, upload the sniffer.tar.gz file on CMS.

If you have any problems about submission, please contact the TAs.

Tips

Again, we strongly recommend you to read Linux Device Driver book. If you crash your machine, do not panic! Carefully look at the dump message (using dmesg) to identify what caused the crash. Then, reboot your instance with euca-reboot-instances command.

CS 5413: High Performance Systems and Networking (Fall 2014) Lab3 - Packet Filter / Sniffer Framework