Blog
Research notes, tutorials, and longer-form writing.
All posts
- Research notes Rethinking Weight Decay From Scalar Shrinkage to Spectral Dynamics
From ordinary weight decay and optimizer evolution to spectral-power decay: exact linear theory, Transformer spectra, and controlled experiments that separate geometric change from training gains.