Scaling Categorical Flow Maps

Apple researchers have introduced a method called Scaling Categorical Flow Maps to improve how language models handle discrete data like text. This approach aims to replace traditional autoregressive models with diffusion-based techniques, potentially enabling faster generation speeds and more flexible control during the sampling process. By adapting flow matching for categorical variables, this research offers a pathway to apply continuous mathematical frameworks to language tasks that were previously restricted to step-by-step prediction methods.