Synopsis Colab TODO Setup What to hand in MNIST Challenge Expected result

Synopsis

In this project, we will be visualizing and manipulating AlexNet [1]:

For this project, we are using PyTorch, an open-source deep learning library that allows an efficient implementation of CNNs. Other similar libraries include Torch, Theano, Caffe, and TensorFlow.

Some parts of this assignment were adapted/inspired from a Stanford cs231n assignment. The parts that are similar have been modified heavily and ported to PyTorch.

The assignment is contained in an IPython Notebook; see below.

[1] Krizhevsky et al, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS 2012

Colab

Google Colab or Colaboratory is a useful tool for machine learning, that runs entirely on cloud and thus is easy to setup. It is very similar to the Jupyter notebook environment. The notebooks are stored on users Google Drive and just like docs/sheets/files on drive, it can be shared among users for collaboration(you'll need to share your notebooks as you'll be doing this in a team of 2).

TODO

To be done in teams of 2.

There are many pieces to the assignment, but each piece is just a few lines of code. you should expect to write less than 10 lines of code for each TODO .

1. Visualize AlexNet structure (TODO 1).
2. Classify Dogs vs Food (TODO 2, 3).
3. Visualize class saliency (TODO 4).
4. Fool AlexNet into making wrong predictions (TODO 5).
5. Visualize a learned class (TODO 6)
6. (4th Credit) Invert AlexNet features (TODO 7a, 7b).
7. (4th Credit) Implement neural net to train on MNIST dataset.

Tests: to verify the correctness of your solutions, you can run tests at very end of the notebook

The instructions about individual TODO are present in detail in the notebooks

Setup

1. Check out the code base from 5670 github repo.
2. There will be two Colab notebooks, Main and MNIST Challenge. If you are doing extra 4th credits upload both of them to your Google drive else just the Main Notebook.
3. Double clicking on the notebook on Google drive should give you an option for opening it in Colab.
4. Alternatively, you can directly open Colab and upload notebook following this:
   File -> Upload notebook...
5. If you haven't used Colab or Jupyter Notebooks before, first read the Colaboratory welcome guide.
6. You will find, rest of the instructions in the notebook. As already stated, Colab required almost no setup, so there is no need to install PyTorch locally.

MNIST Challenge

4-credit students must design and train their own neural network for MNIST dataset. You will be given an example network and your aim should be to improve the accuracy while being under the specified parameter limit. Look at the MNIST notebook for skeleton code and more instructions.

What to hand in?

1. Execute your completed notebook file, download it as ipynb
   File -> Download .ipynb
2. Upload the completed notebooks to CMS
3. If you are doing the MNIST Challenge submit that notebook as well.

What should my images look like?

This section contains images to illustrate what kinds of qualitative results we expect.

Saliency: we expect that pixels related to the class have a higher value. Left: Input image. Right: saliency.

Fooling image

These images look nearly identical, and yet AlexNet will classify each image on the middle as "snail". If you look really closely you can notice some tiny visual differences. The right image shows the difference magnified by 5x (with 0 re-centered at gray).

Class visualization

These images are classified as 100% belonging to different classes by AlexNet. If you run these for longer or adjust the hyperparameters, you may see a more salient result.

Many classes don't give very good results; here we show some of the better classes.

strawberry	throne	mushroom
tarantula	flamingo	king penguin
goblet	sax	llama
cloak	moped	indigo bunting
bulbul	squirrel monkey	cock

Feature inversion (Extra 4th Credits; Optional)

Note that we could probably obtain higher quality reconstructions if we ran the optimization for longer, or added a better regularizer. To keep things simple, your images only need to be mostly converged.

original	conv1	conv2
conv3	conv4	conv5
fc6	fc7	fc8