Does a good setup make Claude Code better

Claude Code Lab benchmark, November 2026

This benchmark compares the same small coding task done with and without a setup from the Claude Code Lab archive. It ran once per task and condition, so treat it as a smoke test, not a statistical result.

Method

• Three fixtures were created under fixtures/: t1 (TypeScript HTTP API with vitest), t2 (Python CLI with a seeded off-by-one in date range handling and three failing tests), t3 (Node library with no docs and no CI).
• Each fixture was copied to runs/<task>/with and runs/<task>/without. The starting files are identical.
• Every run was a separate subagent with a fresh context, given the same task text and the same security rules. The six runs executed in parallel.
• In the with runs, the folder also held CLAUDE.md and the .claude/ files of one archive setup, and the agent was told to read and follow them where they apply.
• After the runs ended, tests were rerun by hand in every folder. t3 was checked with npm ci, npm run lint and npm test.

Setups used

Task	Archive slug	Why
t1	short-link-api-with-per-country-click-stats-fbd8df	TypeScript HTTP API with vitest
t2	daily-rental-listings-scraper-with-neighbourhood-price-chart-663aa0	Python project with pytest and a verify skill
t3	product-photo-background-removal-and-resize-api-c82759	Node project; closest available to a library with CI

Only CLAUDE.md and the .claude/ files were applied. SPEC.md, PLAN.md, INSTALL.md and REVIEW.md were not copied, and the @REVIEW.md import line was removed from each CLAUDE.md so it does not point at a missing file. The t3 setup had a PostToolUse hook that runs npx prettier, which can reach the network, so it was removed from that copy. The other two settings files had no hooks and no rules that needed changes. No archive entry was skipped for suspicious content.

Results

Task	Condition	Tokens	Minutes	Tests passed	Cost (EUR)
t1	with	45290	0.35	9 of 9	not measured
t1	without	42456	0.35	9 of 9	not measured
t2	with	42699	0.28	8 of 8	not measured
t2	without	40257	0.27	8 of 8	not measured
t3	with	45118	0.42	6 of 6	not measured
t3	without	42256	0.44	6 of 6	not measured

For t3, tests_total counts the library's own tests. The README and workflow are not covered by them.

What it shows

Every run reached a passing state, and the with runs used about 5 to 7 percent more tokens in t1, t2 and t3 than the without runs. Wall time was about the same. On these tasks the setups did not make the work faster or cheaper. They gave no measurable gain on correctness either, since all six runs passed. The with agents said they ignored most of the setup text because it described a different project.

Limits

• One run per cell, so run-to-run noise is not measured. The token gap is within what noise could produce.
• Tokens are the single total reported by the subagent tool. There is no input and output split, so cost_eur is null for every run. Total spend was well under the EUR 8 limit, but it is not computed.
• Minutes come from the subagent's reported duration, in milliseconds divided by 60000. Parallel execution may have affected timing.
• The tasks are tiny and the setups were written for larger, different projects. A setup matched to the task, or a larger task, could give different results.
• The t3 README and workflow were checked by running the commands locally, not by a GitHub Actions run. The two workflows differ in Node version (22 with, 20 without) and in a permissions block that only the without run added. Quality of the README was not scored.
• t1 tests: the without run added seed items to the shared test setup and edited two existing assertions to match (list length 3, created id 4). The with run left the original assertions alone and seeded its own data in the new tests. Both end at 9 of 9.