OpenAI and Broadcom unveil LLM-optimized inference chip

OpenAI and Broadcom have unveiled Jalapeño, a custom semiconductor specifically engineered to handle the compute-heavy requirements of running large language models. By shifting from general-purpose hardware to specialized silicon, OpenAI aims to improve the performance and scalability of its AI services. The companies expect this hardware to reduce inference costs by 50 percent, marking a strategic move to lower operational expenses and decrease reliance on external chip suppliers for their core infrastructure.