← Back to Model Beat
Products·5d ago·all news from November 17, 2026

Presentation: Scaling LLM-Based Ranking Systems for Latency-Critical Search & Recommendation Workloads

LinkedIn engineer Sundara Raman Ramachandran detailed techniques for integrating large language models into high-traffic search and recommendation systems while maintaining strict latency requirements. By distilling a 7B model into a 400M parameter version and utilizing single-token embeddings for job contexts, the team achieved a 10x increase in throughput. These methods demonstrate how companies can deploy complex AI ranking models within production environments that demand near-instant response times.

Covered by 1 source

Related stories

ProductsGoogle warns that fully automated cyberattacks have arrived and will get worse fastNov 15 · 23 sourcesProductsIntroducing ChatGPT WalletNov 13 · 6 sourcesProductsApple joins the race for AI agents that control your device while raising new barriers for competitorsNov 19 · 6 sourcesProductsStrengthening NATO and allied cyber defenses with AINov 17 · 2 sources