Search & filter
Filter the full list by keyword, category, or tags. Matching is instant in your browser.
Archive
- Research notes Rethinking Weight Decay From Scalar Shrinkage to Spectral Dynamics
From ordinary weight decay and optimizer evolution to spectral-power decay: exact linear theory, Transformer spectra, and controlled experiments that separate geometric change from training gains.