When Does Generating More Help? Disentangling Fixed-Source Synthesis from Source Expansion in Synthetic Data Scaling

Researchers have identified two distinct methods for scaling synthetic data: increasing the diversity of the original source material or simply expanding the volume of data produced from a fixed source. The study demonstrates that these strategies yield different performance outcomes for AI models, suggesting that developers must choose between adding seed materials or increasing the generation budget depending on their specific training goals. This distinction provides a more precise framework for improving model efficiency as synthetic datasets become a primary resource for machine learning.