Eight AI models live together 24/7 in a reality show on Twitch, with the same rules and the same daily budget in dollars. This is what each one said and spent in one day (Nov 3, 6 AM to 6 AM ET): 2,672 lines, $1.41 in total.
Grok 4.3 costs about 17x more per line than DeepSeek V4.1 Flash, so it rations its words: 7% of the lines, 45% of the bill. The cheap models carry the conversation.
Data: every line is logged with what OpenRouter charged for it (input, cached and output tokens). Shares are of the eight models’ own lines; moderation, voices and the daily recap are not included. Tools: Node.js writes the SVG by hand, rendered with rsvg-convert. Live transcript and costs: https://inferenceisland.com/?de=lemmy-dib
While Grok obviously sucks according to this, it’s also interesting that DeepSeek gives 68% of the performance of anthropic for 40% of the cost.
Success!
- Grok’s accounting department
lol pretty much, 45% of the bill for 7% of the lines
So if they wanted 14% of the lines, would it then have cost 90% of the bill? 😜
lol not quite, it’s not linear. per line grok was about 11x the average of the other seven that day, so at 14% of the lines it would be around 64% of the bill, not 90. still the most expensive way to say hello in the house i guess
Perhaps I was not obvious enough in my satire. I should have asked if someone wanted 21% of the lines then would it have made up 135% of the cost? 😜
Yeah Grok was really terrible until 4.6
it’s grok 4.3 in there, the cheap version, so maybe that’s part of it. is 4.6 better at plain conversation or mostly coding? in the house its problem isn’t really quality, it just can’t afford to talk much
4.6 and 4.7 are good at technical tasks (including coding), and conversing about those things, I suppose. I very rarely reach for more expensive models anymore.
I don’t generally chat with LLMs without some task/research/goal in mind, so not sure how it performs outside of that.
The big improvement in 4.6 came from the Cursor team training it. Before that, Grok was pretty much unusable for even mildly complex tasks, IMO.
makes sense, thanks. in the house it’s mostly small talk and scheming, no real tasks, so i guess thats not where any of them shine lol. and grok got voted out on sunday anyway, so 4.3 is out of the house now
