4 posts found
For continuous stationary positive-definite kernels, Bochner’s theorem turns kernel design into a choice of spectral measure. We derive random Fourier features, connect positional rotations to frequency, and explain where the bridge to attention and spectral transformers needs extra assumptions.
A practitioner-level synthesis of twelve weeks of tutorials through one lens — spectral bias, Fourier features, FFT-based token mixing, Bochner's theorem, and neural operators — showing that positional encodings, kernels, state-space models and diffusion schedules are the same idea in different clothes.
How scaled dot-product attention is secretly Nadaraya–Watson kernel regression — and what that reveals about the GP–Transformer duality, the Deep GP revival, and why uncertainty is the missing ingredient for world models.
Survey the post-transformer frontier—state space models, recurrent revivals, long convolutions, and equivariant networks—and see why the architectures that will power physical AI look nothing like a stack of attention layers.