Limits of Confidence in Diffusion

Apple researchers have identified that discrete diffusion models often struggle with reliability when generating sequences, such as pixels or phonemes, due to how they sample tokens across multiple positions simultaneously. By analyzing these remasking and uniform-state methods, the team demonstrated that current approaches lack sufficient confidence in their output distributions, which can lead to errors in complex data generation. This research highlights fundamental limitations in how these models process discrete data, pointing toward the need for more robust architectural adjustments in generative machine learning.