Back to Blog

Posts tagged “probability-theory”

2 posts found

Probability Theory Explained: Sample Spaces, Random Variables, and the Expectation You Already Optimise
ai-tutorialsbasicsprobability-theory

Probability Theory Explained: Sample Spaces, Random Variables, and the Expectation You Already Optimise

Probability for practitioners who need it for machine learning: why a probability is a function on questions rather than a property of things, why a density can be 40 and is still not a probability, why expectation is linear whether or not anything is independent, why the conditional mean is the best predictor by theorem rather than by training, why cross-entropy's minimiser is calibrated for the same reason, and the seven places — a density mistaken for a probability, base rates, Simpson's paradox, uncorrelated-but-dependent, infinite means, conditioning on a zero-probability event, and 'random' without a measure — where the intuition breaks and the definition does not.

Oct 29, 2026•47 min read
The Probability Domain: Measure, Convergence, and Which of Your Training Claims Are Theorems
ai-tutorialstutorialprobability-theory

The Probability Domain: Measure, Convergence, and Which of Your Training Claims Are Theorems

A standalone tutorial, not a survey: probability as a measure and the σ-algebra as the list of questions you may ask, why that is forced (Vitali) and what it buys (conditioning, filtrations, independence as statements about σ-algebras), a random variable as a measurable function and a density as a Radon–Nikodym derivative with KL's infinities and the GAN support problem falling out, the three integral-limit theorems that license every gradient of an expectation and the one loss where they fail, conditional expectation as the L2 projection that makes the regression function optimal by theorem and the Borel–Kolmogorov paradox computed both ways, the four modes of convergence with the typewriter and escaping-mass counter-examples and the law of the iterated logarithm as the envelope a running mean cannot escape, the central limit theorem as a Fourier theorem with its Berry–Esseen rate and the infinite-width network as a CLT, concentration of measure from Markov to Gaussian Lipschitz with the geometry of d = 4096 and the eval-set error bar table, martingales under SGD, peeking and anytime-valid tests, ergodic theorems under MCMC, Langevin and diffusion as SDE reversal, minimax risk and the price of a density in high dimension, conditional independence as architecture and de Finetti as the licence for Bayesian inference, eleven checkpoint and three diagnostic exercises, and an appendix that grades every connection to our earlier posts by how much it actually proves.

Oct 22, 2026•328 min read