What We Learned by Reproducing 2,200 papers from ICML

Hugging Face researchers attempted to reproduce the code for 2,200 papers presented at the International Conference on Machine Learning, finding that only 54 percent provided usable code. The study highlights significant gaps in software accessibility and standardization within the academic community. This lack of reproducibility creates a barrier for researchers attempting to build upon existing models or verify scientific claims. The findings suggest that the field needs stronger requirements for code sharing to ensure that published advancements remain verifiable and useful for future development.