In the style of Susan Rigetti’s classic “So You Want to Learn Physics”, this post lists some of my favorite resources for learning stuff.
Statistics
- Statistical Modeling: A Fresh Approach (Daniel Kaplan) is a great introduction to frequentist statistics with as little math is possible.
- Regression and Other Stories (Andrew Gelman et al) is similar, but uses a Bayesian approach.
- Statistical Inference (George Casella and Roger L. Berger) is great for learning about estimation and hypothesis testing.
- Plane Analysis to Complex Questions (George E. Forsythe and Cleve B. Moler) is great for learning about analysis of variance if you already know linear algebra.
- Generalized Linear Models with Examples in R (Peter Dunn et al) tells you all about exponential dispersion models and how the residuals we know from linear regression get generalized as “deviances” in GLMs.
- Statistical Rethinking (Richard McElreath) is a great introduction to Bayesian statistics with a focus on modeling.
- Bayesian Data Analysis (Andrew Gelman et al) goes into more detail. It’s the definitive text on Bayesian methods.
- All of Nonparametric Statistics (Larry Wasserman) gives an accessible introduction to nonparametric methods (bootstrap, jackknife, kernel smoothing) and why they work (influence functions, Hadamard differentiability).
- Complex Surveys: A guide to Analysis (Thomas Lumley) talks about estimating statistics of finite populations, which requires different kinds of estimators than those we’ve seen so far.
- Survival Analysis: Techniques for Censored and Truncated Data (John P. Klein and Melvin L. Moeschberger) got me up to speed on survival analysis.
- Causal Inference: The Mixtape (Scott Cunningham) is a great resource for observational studies, with plenty of worked examples.
- Handbook of Markov Chain Monte Carlo is great for understanding and diagnosing failures of fancier MCMC techniques.
- Elements of Sequential Monte Carlo (Christian Naesseth el al) is the simplest tutorial on sequential Monte Carlo methods I’ve seen.
Machine Learning
- Stanford’s CS231n course is a great introduction to deep learning.
- MathematicalMonk’s machine learning videos on YouTube are good for older ML algorithms (the kind in scikit-learn).
- Pattern Recognition and Machine Learning (Christopher M. Bishop) is my go-to reference for machine learning fundamentals.
- Understanding Machine Learning: From Theory to Algorithms (Shai Shalev-Shwartz and Shai Ben-David) gives you a taste of learning theory.
- Gaussian Processes for Machine Learning (Carl Rasmussen and Christopher K. I. Williams) is the classic text on Gaussian processes.
- Reinforcement Learning: An Introduction (Richard S. Sutton and Andrew G. Barto) is a very accessible introduction to reinforcement learning.
- Bandit Algorithms (Tor Lattimore and Csaba Szepesvári) is a comprehensive guide to multi-armed bandit problems.
- Online Learning and Online Convex Optimization (Shai Shalev-Shwartz and Yoram Singer) is a short introduction to online learning.
- Convex Optimization (Stephen Boyd and Lieven Vandenberghe) is the canonical reference for convex optimization. Not exactly ML, but extremely relevant.
Algorithms and Data Structures
- Algorithm Design (Tardos)
- Randomized Algorithms
- Planar Graphs
- All MIT’s open courseware taught by Erik Demaine is awesome. That includes 6.006 Introduction to Algorithms and 6.851 Advanced Data Structures.
- Graph algorithms in the language of linear algebra.
- Spectral graph theory.
Compilers & Programming Languages
- Implementing Functional Languages
- Purely Functional Data Structures
- Types and Programming Languages
- Calculus of Computation
- HOTT
- The Catsters have a YouTube channel on category theory
Math
- Math for Computer Science
- Nonlinear dynamics and chaos
- Visual Group Theory
- 7 Sketches
- Visual Complex Analysis
- Infinite Napkin.
- “The Bright Side of Mathematics” is a YouTube channel with good videos on measure theory.