Regularity and Stability Properties of Selective SSMs with Discontinuous Gating

Researchers have identified that the gating mechanisms in selective state-space models, such as Mamba, can lead to potential numerical instability when processing long sequences. The study provides a formal analysis of these models' coefficients, offering new conditions to ensure stable training and consistent performance for architectures that rely on dynamic, token-dependent gating.