7  Probability

I assume that you have seen some probability theory before, and that this is just a reminder. If you need a more thorough refresher, the book by Ross (Ross 2014) is a popular introductory text that covers discrete and continuous problems, but not more general probability measures. Another good undergraduate text by Chung and AitSahlia (Chung and AitSahlia 2003) includes a little bit of measure theory. Good graduate texts include the books by Billingsley (Billingsley 1995) and by Breiman (Breiman 1992). If you want a reminder that is more thorough than the one we give here, but less than a full textbook, the treatment in (Deisenroth, Faisal, and Ong 2020) is a good starting point.

• Axiomatic probability, counting, and measure

• Conditional and marginal probabilities

• Bayesian and frequentist perspectives

• Random variables

• The wonder of Gaussians (and not all RVs are Gaussian!)

• Moments (+ linearity of expectations, variance and precision,heteroscedasticity, etc) and tails

• Independence, conditional independence, factorization of probabilities, graphical models

• Standard statistics (incl variance and precision)

• Central limit theorem / LLN / sums

• Parameter estimation

• Bayes and updating

• Conjugate priors

• Uninformative priors

• Markov chains

• Martingales

• Stochastic processes

• Standard inequalities and concentration of measure

Billingsley, Patrick. 1995. Probability and Measure. Third. Wiley Series in Probability and Mathematical Statistics. Wiley.

Breiman, Leo. 1992. Probability. SIAM. https://doi.org/10.1137/1.9781611971286.

Chung, Kai Lai, and Farid AitSahlia. 2003. Elementary Probability Theory. Undergraduate Texts in Mathematics. Springer. https://doi.org/10.1007/978-0-387-21548-8.

Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. 2020. Mathematics for Machine Learning. Cambridge University Press. https://mml-book.com.

Ross, Sheldon. 2014. A First Course in Probability. Ninth. Pearson.