← Back to Model Beat
Research·Nov 10·all news from November 10, 2026

Toward a Theory of Evaluability

Machine learning models, particularly large foundation models, are increasingly evaluated along multiple axes that are relevant for their usage. In addition to accuracy measures appropriate for the specific task of interest, measures capturing safety, calibration and fairness properties are often of interest. The classical approach to these problems assumes that the test set has been held out from the training process, an assumption that is increasingly harder to justify for large foundation models. In this work, we formalize a notion of evaluability without this assumption, and ask if it is…

Covered by 2 sources

Related stories

ResearchAlphaGenome 2: Deciphering how cells read DNA to better understand diseaseNov 11 · 4 sourcesResearchWork at the Frontier: AI’s effect on freelance workNov 10 · 2 sourcesResearchUS data brokers are selling half a billion dollars in RL training environments to Chinese AI labsNov 11ResearchSupporting research on AI and children’s safetyNov 12