Back to Blog

Posts tagged “monte-carlo”

2 posts found

Concentration and the Central Limit Theorem Explained: How Many Samples Your Number Needs
ai-tutorialsbasicsprobability-theory

Concentration and the Central Limit Theorem Explained: How Many Samples Your Number Needs

The error bar for practitioners: why an average of independent, identically distributed draws has a Gaussian error of size sigma over root n and how fast that becomes true, why a mean of rare events needs a hundred times more data than a symmetric one, what Chebyshev, Hoeffding and the exact binomial each know, how many test examples resolve a one-point gap (about 12,000 unpaired, a few hundred disagreements paired), why picking the best of a hundred checkpoints adds three points you did not earn, why random vectors in 4,096 dimensions are nearly orthogonal and attention divides by root d, and why 10,000 correlated samples can be worth 50 independent ones — with the batch-means fix.

Oct 31, 2026•56 min read
The Probability Domain: Measure, Convergence, and Which of Your Training Claims Are Theorems
ai-tutorialstutorialprobability-theory

The Probability Domain: Measure, Convergence, and Which of Your Training Claims Are Theorems

A standalone tutorial, not a survey: probability as a measure and the σ-algebra as the list of questions you may ask, why that is forced (Vitali) and what it buys (conditioning, filtrations, independence as statements about σ-algebras), a random variable as a measurable function and a density as a Radon–Nikodym derivative with KL's infinities and the GAN support problem falling out, the three integral-limit theorems that license every gradient of an expectation and the one loss where they fail, conditional expectation as the L2 projection that makes the regression function optimal by theorem and the Borel–Kolmogorov paradox computed both ways, the four modes of convergence with the typewriter and escaping-mass counter-examples and the law of the iterated logarithm as the envelope a running mean cannot escape, the central limit theorem as a Fourier theorem with its Berry–Esseen rate and the infinite-width network as a CLT, concentration of measure from Markov to Gaussian Lipschitz with the geometry of d = 4096 and the eval-set error bar table, martingales under SGD, peeking and anytime-valid tests, ergodic theorems under MCMC, Langevin and diffusion as SDE reversal, minimax risk and the price of a density in high dimension, conditional independence as architecture and de Finetti as the licence for Bayesian inference, eleven checkpoint and three diagnostic exercises, and an appendix that grades every connection to our earlier posts by how much it actually proves.

Oct 22, 2026•328 min read