The Mathematics of Large Language Models: Machine Learning Theory Made Readable: LLMs, Transformers, Diffusion, Neural Networks, Optimization, and Generative AI

Revised and updated,August 2026

Most explanations of artificial intelligence stop just before the mathematics becomes
interesting. This book goes further.

Books about AI usually take one of two approaches. They avoid the equations entirely, or they present them as if you already speak the language. This one does neither. The mathematics is here in full, unsimplified, and so is a way to read it.

It is written for people who want the actual mathematics behind these systems and who keep getting stopped by the notation. That is a real and common place to be stuck, and it is not the same thing as being unable to follow the argument. No advanced degree is assumed. What is assumed is that you are willing to sit with an equation until it opens.

How the mathematics is unpacked

After every equation, two short passages do the work:

What It Does explains, in plain language, what the formula is for.

Reading the Formula walks through the notation symbol by symbol: what each part contributes, what changes when you alter it, and why the equation is written the way it is.

These do not simplify the mathematics. The expression on the page is the one the field uses, at full strength, because a watered-down version would teach you something that is not true. What the two passages give you is a way to climb up to it.

How the book is built
Seventeen chapters, and they are not seventeen separate surveys. A preface lays out the
structure before you start: which chapters stand alone, which ones the rest of the book leans on, and several routes through depending on what you came for. Every chapter then opens by saying what it establishes, what it assumes, and what later chapters build on it. You always know where you are and why you are there.

The stories
Each chapter carries the human story behind the ideas in it: the observation, the failed
experiment, the competition result, the unexpected connection that made researchers rethink how learning works. The mathematics arrives attached to the problem it was invented to solve, which is how it was actually discovered and how it is easiest to hold onto.

What the book covers
Approximation and what a network can represent. Optimization and what training can actually find. Generalization, implicit bias, and double descent. Symmetry and convolution. Recurrence and state-space models. Attention and transformers. Graphs. Latent-variable and adversarial models. Diffusion and optimal transport. Continuous-depth models. Operator learning for scientific problems. Bayesian methods and calibrated uncertainty. Robustness and causality. Scaling laws and in-context learning. And a closing chapter on hallucination: what the mathematics says about what a model cannot know, and when abstaining is the correct answer.

One honest note on fit

If you already read this notation fluently, the explanatory apparatus will be in your way. You
are welcome here, and you should skip it: the results, derivations and citations stand without it. But the scaffolding is the point of this book, and it was built for readers who need it.

If you have ever wanted to move past surface-level explanations and understand the mathematics that makes modern AI work, this book was written for you.

Jason Karpeles is an award winning data scientist and predictive-analytics innovator with thirty years building forecasting and machine-learning models in industry. Jason earned a master’s degree in economics from NYU and an MBA from Duke University. Full biography under About the Author.