Hyperball
During training, not only which connections matter changes, but also the overall magnitude of their weights — numbers controlling how connections influence the result. You want to separate changes in the direction of these numbers from changes in their scale. Slowly shrinking weights alone does not strictly control their magnitude.
Hyperball updates selected weight matrices so they retain a specified norm, a measure of overall magnitude. Imagine all the numbers in one matrix as a point's coordinates. The point can move on a sphere's surface while its distance from the center stays fixed.
In the original method, §3, the proposed change is also normalized, and weights are restored to the sphere after the step. The model still learns: proportions between numbers change while their shared Frobenius norm remains fixed.
This is an analogy for the geometry of many numbers, rather than a physical ball inside the computer. The method controls the norm more directly than Weight decay. Results depend on training design and settings; keeping a fixed norm alone does not guarantee better answers.