Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment

Researchers from Apple published a study evaluating how well large language models perform as judges when tasked with grading text based on a user's specific writing style preferences. The paper highlights the potential challenges of using automated systems to assess subjective qualities like tone, suggesting a need for more reliable frameworks when deploying AI to provide feedback on personalized content.