The Probability Domain: Measure, Convergence, and Which of Your Training Claims Are Theorems
A standalone tutorial, not a survey: probability as a measure and the σ-algebra as the list of questions you may ask, why that is forced (Vitali) and what it buys (conditioning, filtrations, independence as statements about σ-algebras), a random variable as a measurable function and a density as a Radon–Nikodym derivative with KL's infinities and the GAN support problem falling out, the three integral-limit theorems that license every gradient of an expectation and the one loss where they fail, conditional expectation as the L2 projection that makes the regression function optimal by theorem and the Borel–Kolmogorov paradox computed both ways, the four modes of convergence with the typewriter and escaping-mass counter-examples and the law of the iterated logarithm as the envelope a running mean cannot escape, the central limit theorem as a Fourier theorem with its Berry–Esseen rate and the infinite-width network as a CLT, concentration of measure from Markov to Gaussian Lipschitz with the geometry of d = 4096 and the eval-set error bar table, martingales under SGD, peeking and anytime-valid tests, ergodic theorems under MCMC, Langevin and diffusion as SDE reversal, minimax risk and the price of a density in high dimension, conditional independence as architecture and de Finetti as the licence for Bayesian inference, eleven checkpoint and three diagnostic exercises, and an appendix that grades every connection to our earlier posts by how much it actually proves.
