The inventor of the bitter lesson has a bitter lesson for LLM scaling enthusiasts

Turing Award recipient Richard Sutton and former OpenAI researcher Lukasz Tworek are challenging the current industry focus on scaling transformer-based models with raw compute. They argue that this approach has become a substitute for genuine innovation and that achieving artificial general intelligence will require the development of fundamentally new learning methods rather than just larger datasets. Their critique suggests that the field risks stagnation if it continues to prioritize computational output over architectural breakthroughs.