What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

Researchers investigating neural text degeneration have published a study comparing how training data versus model architecture contributes to repetitive output. The paper evaluates whether repeating content stems from exposure to repetitive datasets or from internal mechanisms within the network itself. By using a cross-model fixed-point census, the authors aim to distinguish between these two competing explanations for why language models frequently get stuck in repetitive patterns.