Quantitative Analysis of Sentiment Expression Across Large Language Models: A Comparative Study Using Plutchik’s Wheel of Emotions

Link to paper: https://zenodo.org/records/14344936
Publication date: 12/10/2024

Recent advances in Large Language Models (LLMs) have dramatically transformed the landscape of natural language processing, yet our understanding of how these models express and manipulate emotional content remains limited. This study presents a comprehensive analysis of sentiment expression across multiple prominent LLMs, including Llama 8B, Gemini 1.5 Flash, ChatGPT 4, and Claude 3.5 Sonnet. Using Plutchik’s Wheel of Emotions as a theoretical framework, we evaluate how different LLMs express and combine emotional states through generated text. Our analysis employs both LIWC (Linguistic Inquiry and Word Count) and SALLEE (Syntax-Aware LexicaL Emotion Engine) to quantify emotional expression across 50 text generations per sentiment per model. Results reveal distinctive patterns in how different LLMs handle emotional intensity and emotional combinations, with significant variations in consistency and accuracy across models. These findings have important implications for both practical applications of LLMs and theoretical understanding of artificial emotional expression.

Creators

• Butler, Raleigh (Project manager)
• Ward, Dylan (Project member)
• Jenkins, Dana (Project member)
• Lantrip, A.R. (Project member)
• Armstrong, Erin (Other)
• Butler, Rory (Other)
• Driza, Paige (Other)
• Fields, Jackson (Other)
• Levario, Ricardo (Other)
• Miller, Kylee (Other)
• Plessala, Bennett (Other)
• Sigman, Nathaniel (Other)
• Slater, Leah (Other)
• Vassallo, Emily (Other)
• Vivekanandan, Avinash (Other)
• Yildirim, Lisa (Other)