Writing & Editing Typetone (WETT) 
LLM Benchmark


We compared performance of major LLMs on writing and editing tasks.
So that you can pick the one most suitable for your creative writing needs.
Models ranked from best to worst on average of all tasks
Writing instructions adherence
Stylistic and vocabulary diversity
Self-evaluation capability
The amount of time to onboard someone for content marketing, back and forth, feedback.
o3-mini
 94.7%
89.8%
         2.024
81.3%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Claude Sonnet 3.5 (Oct. 2024 model)
      91.6%
89.6%
2.103
         78.7%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Qwen 2.5-72B
      90.6%
         80.8%
         1.988
84.5%
The amount of time to onboard someone for content marketing, back and forth, feedback.
GPT 4o (Nov. 2024 model)
      92.0%
         75.4%
2.083
         79.4%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Claude Sonnet 3.7
      91.9%
         86.4%
         1.932
         76.8%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Gemini 2.0 Flash-Lite
 92.5%
         86.0%
1.900
         74.5%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Qwen 2.5-7B
      89.9%
89.8%
         1.992
44.9%
The amount of time to onboard someone for content marketing, back and forth, feedback.
GPT 4o-mini
 92.4%
         77.4%
         2.055
39.0%
The amount of time to onboard someone for content marketing, back and forth, feedback.
DeepSeek V3
      90.1%
         81.2%
         2.024
         78.3%
The amount of time to onboard someone for content marketing, back and forth, feedback.
GPT 4o (Aug. 2024 model)
 84.2%
54.6%
         2.079
         80.9%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Llama 3.1-405B
      85.6%
         80.3%
         2.021
         79.0%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Mistral Small 3
 75.6%
         76.6%
         2.046
         80.7%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Gemini 1.5 Pro
      89.3%
71.4%
         1.988
         80.6%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Gemini 2.0 Flash
      86.5%
         81.6%
         1.980
         71.4%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Claude Haiku 3.5
      92.1%
         77.8%
1.919
         69.2%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Gemini 1.5 Flash
      86.9%
         73.2%
1.927
80.9%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Llama 3.3-70B
      88.1%
         80.6%
         1.938
         65.5%
The amount of time to onboard someone for content marketing, back and forth, feedback.
Llama 3.1-8B
 82.4%
64.8%
2.086
50.4%

Main author: Azamat Omuraliev
Collaborators: Sam Baud, Sjoerd de Kreij, Jirmaine Tan, Mayuran Visakan, Grzegorz Lazka, Yun Yang, Jeroen Meij

Read about the methodology in our blogpost

Risk free try out

🚀 Discover how Sarah can transform your marketing in just 15 minutes 🔥 Limited slots available - Don't miss out!

Grow on autopilot, get unburdened by Sarah.

Schedule a call