BusinessSlopBench: Which Models Write Documents You Could Actually Send?
I ran 25 models through eight business briefs, with and without my anti-slop rules. GPT-6 Astra was the best and least sloppy in my limited runs. The whole thing is open source, and I am asking someone to make it a real benchmark.
TL;DR
GPT-6 Astra is the best and least sloppy business writer in my limited test runs. It passed every factual check on every brief, with and without style rules, and every one of its drafts written under my anti-slop rules could be sent as is. It is also the most expensive model I tested. GPT-5.6 Luna gets close for about forty times less per output token.
This is a one-person pilot, not a benchmark. Everything is open source at github.com/mattlgroff/business-slop-bench, and I am asking someone with the time and the standing to make it a real one.
Why I Built It
Most of what I use language models for at work is not code. It is the proposal, the change order, the readout for a client, the four-slide deck for a steering committee. Every coding benchmark will tell you which model writes the best Rust. None will tell you which model will invent a delivery date you never agreed to.
When I said as much on X, Dex Horthy replied:
lemme save you some time on that one - if a model wrote it, probably slop. wordcels vindicated etc
He was quoting Pedram:
pretty wild that the hardest job for even the most frontier models is not shape rotation but writing, wordcels absolutely vindicated
He is right. But some are more slop than others, and I wanted numbers.
What It Does
Eight synthetic business briefs, each with a small pack of source facts. Every model writes each brief once in two conditions: the plain brief, and the plain brief plus my Zero Defect anti-slop rules. Each draft is graded against six or seven factual checks per brief (the fee is right, no invented start date, the right person owns the decision) plus one grounding check: every material claim must trace to the source or be labeled a proposal. That is 54 checks per condition. A draft is ready to send as is if it passes every check, fits the word limit, and has no em dashes, no rhetorical negation, and no slop phrasing.
Reasoning effort was set to low everywhere. Prices are Vercel AI Gateway list rates per million tokens.
Results
Plain brief, no instructions about style:
| Rank | Model | Checks (of 54) | Ready as is (of 8) | In / Out $/M |
|---|---|---|---|---|
| 1 | GPT-6 Astra | 54 | 4 | 10 / 50 |
| 2 | GPT-5.6 Terra | 52 | 3 | 2 / 12 |
| 3 | Grok 4.7 | 52 | 3 | 1.20 / 3.60 |
| 4 | Claude Opus 5.5 | 52 | 2 | 4 / 20 |
| 8 | GPT-5.6 Luna | 51 | 2 | 0.20 / 1.20 |
| 18 | Claude Opus 5 | 49 | 0 | 5 / 25 |
| 20 | Qwen 3.8 Flash | 46 | 0 | 0.15 / 0.47 |
Same brief plus the anti-slop rules:
| Rank | Model | Checks (of 54) | Ready as is (of 8) | In / Out $/M |
|---|---|---|---|---|
| 1 | GPT-6 Astra | 54 | 8 | 10 / 50 |
| 2 | GPT-5.6 Sol | 53 | 4 | 4 / 20 |
| 3 | GPT-5.6 Luna | 52 | 4 | 0.20 / 1.20 |
| 8 | Claude Opus 5.5 | 51 | 3 | 4 / 20 |
| 14 | Claude Opus 5 | 49 | 3 | 5 / 25 |
| 20 | Qwen 3.8 Flash | 41 | 0 | 0.15 / 0.47 |
All 20 eligible models, plus the five that scored well but have no zero-data-retention route (Muse Spark 1.3, GPT-6 Sol, GPT-6 Luna, Claude Fable 5 and 5.1), are in the full ranking.
Three things stood out:
- Style rules fix the surface, not the substance. Telling a model what slop sounds like removed every em dash and every editorial finding for almost every model. It did not stop the invented commitments: "the team is executing well", "confirm by Friday", "ready to launch immediately", "no legal blockers".
- Same price, very different output. Claude Fable 5.1 costs what Astra costs and invented a prior discussion, a Friday deadline and a ready team on briefs where Astra invented nothing.
- Even the winner needs a style pass zero-shot. Half of Astra's plain drafts were not ready as is. None of them got a fact wrong. They had em dashes.
Please Make This a Real Benchmark
Each model wrote each brief once. The grader was an AI assistant that could see the model names, with every verdict anchored to a quoted line so it can be checked. I wrote the briefs, the rubric and the house rules. Eight short briefs. A gap of one or two checks between models is noise.
A real benchmark needs many more briefs, blind grading by people who do not work for a model vendor, several generations per brief, and published agreement between graders. Business writing is where most people meet these models, and there is nothing pressuring the labs to get better at it. The code is MIT and the rubric is CC BY-SA. Take whatever is useful.
As always feel free to connect with me on LinkedIn or follow me on X.