Synopsis Colab TODO Setup What to hand in MNIST Challenge Expected result

Synopsis

In this project, we will be visualizing and manipulating AlexNet [1]:

For this project, we are using PyTorch, an open-source deep learning library that allows an efficient implementation of CNNs. Other similar libraries include Torch, Theano, Caffe, and TensorFlow.

Some parts of this assignment were adapted/inspired from a Stanford cs231n assignment. The parts that are similar have been modified heavily and ported to PyTorch.

The assignment is contained in an IPython Notebook; see below.

[1] Krizhevsky et al, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS 2012

Colab

Google Colab or Colaboratory is a useful tool for machine learning, that runs entirely on cloud and thus is easy to setup. It is very similar to the Jupyter notebook environment. The notebooks are stored on users Google Drive and just like docs/sheets/files on drive, it can be shared among users for collaboration(you'll need to share your notebooks as you'll be doing this in a team of 2).

TODO

To be done in teams of 2.

There are many pieces to the assignment, but each piece is just a few lines of code. you should expect to write less than 10 lines of code for each TODO .

1. Visualize AlexNet structure (TODO 1).
2. Classify Dogs vs Food (TODO 2, 3).
3. Visualize class saliency (TODO 4).
4. Fool AlexNet into making wrong predictions (TODO 5).
5. Visualize a learned class (TODO 6)

Tests: to verify the correctness of your solutions, you can run tests at very end of the notebook

The instructions about individual TODO are present in detail in the notebooks

Setup

1. Check out the code base from 5670 github repo.
2. There will be two Colab notebooks, Main and MNIST Challenge. If you are doing extra credits upload both of them to your Google drive else just the Main Notebook.
3. Double clicking on the notebook on Google drive should give you an option for opening it in Colab.
4. Alternatively, you can directly open Colab and upload notebook following this:
   File -> Upload notebook...
5. If you haven't used Colab or Jupyter Notebooks before, first read the Colaboratory welcome guide.
6. You will find, rest of the instructions in the notebook. As already stated, Colab required almost no setup, so there is no need to install PyTorch locally.

What to hand in?

1. Execute your completed notebook file, download it as ipynb
   File -> Download .ipynb
2. Upload the completed notebooks to CMS
3. If you are doing the MNIST Challenge submit that notebook as well.

What should my images look like?

This section contains images to illustrate what kinds of qualitative results we expect.

Saliency: we expect that pixels related to the class have a higher value. Left: Input image. Right: saliency.

Fooling image

These images look nearly identical, and yet AlexNet will classify each image on the middle as "snail". If you look really closely you can notice some tiny visual differences. The right image shows the difference magnified by 5x (with 0 re-centered at gray).

Class visualization

These images are classified as 100% belonging to different classes by AlexNet. If you run these for longer or adjust the hyperparameters, you may see a more salient result.

Many classes don't give very good results; here we show some of the better classes.

strawberry	throne	mushroom
tarantula	flamingo	king penguin
goblet	sax	llama
cloak	moped	indigo bunting
bulbul	squirrel monkey	cock

MNIST Challenge

Extra credit students must design and train their own neural network for MNIST dataset. You will be given an example network and your aim should be to improve the accuracy while being under the specified parameter limit. Look at the MNIST notebook for skeleton code and more instructions.