Tag: overfitting

All the articles with the tag "overfitting".

• Features and Overfitting: How a Model Memorizes, and How L1 and L2 Stop It

  Updated:24 Oct, 2026

  Give the router 379 features instead of one and it scores 100% on its training data and worse on new questions. Why that happens, why the weights run to infinity, how L2 and L1 penalties stop them (derived, with the soft-threshold that makes L1 exact zeros), why one well-chosen feature matches all 379, and how to tell a real difference from noise when the splits share data.