📚 Book
2024
🇬🇧
Francis Bach
Machine Learning / Data Science / Deep Learning
Learning 💪
This book is a rare attempt to rebuild machine learning theory from the ground up, starting not from fashionable models but from the fundamental questions: what does it mean to learn, under which assumptions and at what cost in data, computation and structure. Francis Bach develops a coherent theory of supervised learning rooted in decision theory, concentration inequalities, optimization and approximation, carefully exposing the trade-offs that govern generalization.
Rather than organizing the field by model families, the book progresses by principles. Classical methods such as linear regression, kernel methods, sparsity and nearest neighbors are treated as canonical case studies through which core ideas emerge: regularization as necessity, assumptions as the true source of learning and rates as the only honest currency. Neural networks, ensembles and overparameterized models are analyzed in deliberately simplified settings where estimation, optimization and implicit bias can be understood precisely: this dispels the myth that parameter count alone explains generalization.
The treatment is unusually unified: convex and non-convex optimization, stochastic gradient descent, variance reduction, online learning, bandits, structured prediction, probabilistic interpretations, PAC-Bayesian analysis and matching lower bounds are all connected through a consistent mathematical language.
Loving 💜
What I admire most is the intellectual honesty of this book, encouraging me to ask fundamental questions that sharpen both understanding and hands-on intuition. There is no hand-waving, no slogan-level intuition and no reverence for models just because they are popular. Every method earns its place through assumptions, proofs and rates. What is rare is that this rigor targets exactly the topics that most theory books still treat as footnotes: overparameterization, implicit bias, modern optimization dynamics. It reshaped how I think about generalization, especially in modern settings where these effects matter more than textbook capacity measures.
This is a book to grow with. For anyone who wants to move beyond recipes and understand why learning algorithms actually work, or provably cannot, this is one of the most demanding and rewarding references available today.