Syllabus | Machine Learning

(Disclaimer, the syllabus is subject to change.)

1. Intro: What is machine learning? Project 0
2. kNN classification Project 1
3. curse of dimensionality
4. Perceptron Project 2
5. MLE MAP estimation
6. Naive Bayes [Project 3]
7. Logistic Regression [ HW#1 due, HW#2 out ]
8. Gradient Descent
9 Linear Classifiers
- Ordinary Least Squares
- Linear Support Vector Machines
- Logistic Regression
- Gradient Descent, Newton's Method
- Empirical Risk Minimization, Loss functions
10 SVEN
11 Bias Variance Tradeoff
- ML Debugging
- Over / Underfitting
12 the kernel trick, kernel machines
- kernel Ridge Regression
- SVM dual
13 Gaussian Processes
- Bayesian Optimization

Future topics:
CART Trees
- Decision Trees
- Regression Trees
Bias Variance Tradeoff
- Tree Pruning, Overfitting, Underfitting, Regularization, Model complexity
- ERM Regularizers
Bagging (reduces variance)
Boosting (reduces bias)
Machine Learning Debugging
Online Learning
- Stochastic Gradient Descent
- Passive Aggressive Learning?
multi-class
- 1 vs all
- 1 vs 1
Deep Learning
- BackPropagation
- Regularization: Dropout, Weight Decay
- Convolutional Neural Networks
- Denoising) Autoencoders
Metric Learning
- LMNN / NCA

Unsupervised Learning:
k-means (clustering),
EM algorithm, GMM

Further topics:
Recommender systems (Netflix), Matrix factorization
Graphical Models, Sparse learning,