Cheap vs Flagship AI Models: $0.10 vs $198

One model built this desktop for 10 cents. Another built it for $198. Same prompt, same harness, same afternoon.

Model cards price tokens, not tasks. The cost of an agentic run is set by how many turns a model takes, and a model that is 30x cheaper per token can still lose if it loops 30 times. So we ran an AI coding model benchmark the honest way: one prompt, one harness, six models, and a stopwatch. Every run went through LLM Gateway, so switching models was one environment variable and the per-model cost landed in one place.

The prompt and the idea come from @bytrishalim on Instagram, who posted the retro girly desktop concept that started this. It turned out to be a genuinely good coding eval.

The Task

Each model got the same brief plus three reference images read off disk:

design a windows retro girly desktop. there needs to be a couple of windows open like winamp player. a paint app. a notepad. sticky note. word document maybe. attached are some inspo

Use a simple HTML + CSS layout to keep it simple. Write the result to index.html (plus a style.css if you want). No build tools, no frameworks, no JS required.

Open-ended but objectively checkable. Every run has to read three images, hold a visual style in mind, hand-write a lot of CSS, and lay out six overlapping windows without clipping them. There is no library that does it for you.

The harness was Claude Code running headless in a clean directory. The only thing that changed between runs was the model:

1export ANTHROPIC_BASE_URL=https://api.llmgateway.io2export ANTHROPIC_AUTH_TOKEN=$LLM_GATEWAY_API_KEY3export ANTHROPIC_MODEL=novita/deepseek-v4.1-flash4export ANTHROPIC_DEFAULT_HAIKU_MODEL=$ANTHROPIC_MODEL5
6claude -p "$(cat prompt.txt)" --permission-mode bypassPermissions

1export ANTHROPIC_BASE_URL=https://api.llmgateway.io2export ANTHROPIC_AUTH_TOKEN=$LLM_GATEWAY_API_KEY3export ANTHROPIC_MODEL=novita/deepseek-v4.1-flash4export ANTHROPIC_DEFAULT_HAIKU_MODEL=$ANTHROPIC_MODEL5
6claude -p "$(cat prompt.txt)" --permission-mode bypassPermissions

Pick any model from the live model directory, set it as ANTHROPIC_MODEL, and the gateway handles protocol translation to whichever provider serves it.

What It Cost

All six runs finished and produced a working index.html plus style.css.

Astra’s figure includes $16.48 of claude-sonnet-5 requests from subagents it spawned for its own review passes — 35 of its 94 requests. With agentic harnesses, the model you set is not always the only model you pay for.

Total for the benchmark: $329.63 across 253 requests and 60 million tokens — of which two runs account for $324.89.

What Each Model Built

Muse Spark 1.3 Contributor — $0.10, 123s

The cheapest run and the best writing of the six. The diary nails the Xanga-era voice — a mood field, "everything is PINK now!!", a guestbook sign-off from glitter_gurl_98 — and Winamp has a real playlist widget. The chrome is the weakest: flat wallpaper, the smallest CSS at 7.2K, everything crowded into the top-left. One shot, no verification.

GLM 5.3 Flash — $0.11, 103s

The fastest run, done in under two minutes across 8 requests with no iteration at all. Properly pinkified Win98 bevels, a pure-CSS Winamp equalizer, an inline-SVG Paint doodle, working taskbar buttons. The middle of the desktop is sparse and Notepad slides behind the taskbar at shorter viewports. For 11 cents, remarkably little is wrong with it.

DeepSeek V4.1 Flash — $1.18, 419s

The first model to check its own work. It opened a browser, found that its layout overflowed the viewport, built a scale-to-fit system to fix it, and re-verified at four widths. The detail shows: a Winamp LCD with seek bar and volume knob, Word with a full status bar (At 4.5" Ln 8 Col 12 REC), Notepad reporting Ln 9, Col 28 · 100% · Windows (CRLF) · ANSI, MSN Messenger in the taskbar. The highest-fidelity artifact that renders correctly at any size.

Gemini 3.8 Flash — $3.35, 916s

The richest artwork per dollar anywhere in the benchmark. Gemini wrote a 47K index.html — more than twice any other model — holding a hand-vectored kawaii bunny under a rainbow, a 16-tool Paint palette with pixel-accurate SVG icons, gingham wallpaper, nine desktop icons and a WordPad with a graduated ruler. It is also the only run with visibly broken layout: Paint covers WordPad's titlebar, Winamp covers Notepad's. Lots of craft, not enough checking.

GPT-6 Astra — $126.98, 2402s

The best-designed result, and the only one that reads as art-directed rather than decorated. Muted mauve and rose instead of saturated pink, a "hello, daydreamer" masthead, a pixel-art heart in Paint, a Winamp playlist editor labelled "the main character mix". The only run with zero window overlap and nothing clipped, verified at five widths down to 320px. It also wrote itself a PRODUCT.md, a 15K DESIGN.md and spawned review subagents — which is exactly why it took 40 minutes and $127.

Claude Fable 5.1 — $197.91, 1045s

The most convincing period artifact, and the single most expensive index.html we have ever generated. The diary is written by an actual teenager in 2003 — "he sent me a smiley face. a SMILEY FACE. i'm gonna die." — Paint holds a bow-wearing cat, the Winamp playlist is a real nine-track Y2K set with runtimes, Word carries the full status bar, and there are three tilted sticky notes including one with a password hint. It caught its own bug mid-run: the playlist had rendered below the taskbar, so it moved it up and raised the minimum canvas size.

Where the $198 Went

Not into code. Input was roughly 99% of tokens in every run. Astra read 22.5 million tokens to produce a 19K HTML file. Fable read 20.8 million to produce 17K.

The expensive models are not expensive because they write more. They are expensive because they take more turns, and every turn resends the accumulated context. GLM finished in 7 requests. Fable took 35. That 5x gap in turns, compounded by a context that grows on each one, is where a 1,979x cost difference actually comes from.

Which also means the cheap-model advantage is bigger than the per-token prices suggest.

The Quality Gap Is Real. It Is Not 1,979x.

The flagships won. Astra and Fable are the two artifacts we would actually ship, and neither budget model is close on craft. But the curve flattens fast:

• Gemini 3.8 Flash at $3.35 — roughly 80% of Fable's visual richness for 1.7% of the cost.
• DeepSeek V4.1 Flash at $1.18 — roughly 75% for 0.6%, and unlike Gemini it renders correctly.
• GLM 5.3 Flash at $0.11 — roughly 60% in 103 seconds with no iteration at all.

If you run a coding agent at any volume, the question is not "which model is best". It is "at what point does the next dollar stop buying anything". On this task that point sits much closer to the cheap end than the price list implies:

• Volume agentic work, price-sensitive: DeepSeek V4.1 Flash or Gemini 3.8 Flash.
• Fast one-shot generation: GLM 5.3 Flash.
• Final output quality, taste-driven: Claude Fable 5.1.
• Final output quality, rigor and responsive correctness: GPT-6 Astra.

Run This Yourself

Every number here came out of the llmgateway CLI, which reads the same per-model aggregates as the dashboard:

1llmgateway usage --by model --range 24h

1llmgateway usage --by model --range 24h

That is the part worth stealing regardless of which models you care about. Point your agent at the gateway, run your own real task, and read the cost per model afterwards instead of estimating it from a price table. The two numbers rarely match.

Caveats, for honesty: each figure is one full agent run per model, attributed by session id, and quality scoring is a judgement call — the screenshots are there so you can disagree with ours.

• Try LLM Gateway free — run any model through one Anthropic-compatible endpoint
• Browse the model directory — live pricing and capabilities for every model in this post
• Track LLM usage and spend — per-model tokens, cache hits and costs via the API
• @bytrishalim on Instagram — for the prompt and the idea behind this benchmark