Rapid LLM development is outrunning the research meant to explain it, study finds

A Stanford study found that nearly 80 percent of research claims about large language models have weakened or become obsolete within an average of 15.7 months. Researchers evaluated 50 common assertions against 20 modern models, discovering that conclusions about AI limitations expire the fastest. This rapid turnover highlights how quickly technical development is outpacing scientific evaluation, complicating efforts to maintain an accurate academic understanding of what current artificial intelligence systems can and cannot do.