Posts tagged “transformers”

3 posts found

Attention Is All You Need... Is a Kernel

How scaled dot-product attention is secretly Nadaraya–Watson kernel regression — and what that reveals about the GP–Transformer duality, the Deep GP revival, and why uncertainty is the missing ingredient for world models.

Jul 12, 2026•12 min read

ai-tutorialstutorialvision-transformer

How Vision Transformers Work (and Why Patch Tokenization Is a Hack)

Understand the elegant trick that let transformers process images, why it surprisingly works, and why its fundamental limitations drive the next generation of vision architectures.

Apr 10, 2026•11 min read

ai-tutorialstutorialpost-transformer

Beyond Attention: The Post-Transformer Architecture Landscape for Physical AI

Survey the post-transformer frontier—state space models, recurrent revivals, long convolutions, and equivariant networks—and see why the architectures that will power physical AI look nothing like a stack of attention layers.

Apr 9, 2026•17 min read