Z.ai’s Jie Tang Shows GLM Data Recipe: 1T Pages, 500M Repos

Z.ai researcher Jie Tang has disclosed that the GLM model was trained on a dataset comprising one trillion web pages and 500 million code repositories. This revelation provides insight into the massive scale of data procurement required to develop large language models, highlighting the industry reliance on vast web-based information and software development platforms.