I Checked the Viral "65% to 94%" Claude Stat

A screenshot has been going around: install this one skill file, and Claude Code goes from 65% to 94% accuracy. It is a great number. It has the shape of evidence. So I opened the repo it points to.

There is no benchmark in it. No eval, no test set, no methodology, no percentages at all. The repository is a set of coding guidelines derived from Andrej Karpathy's public observations about how language models fail at coding — thoughtful, but explicitly not measured. The number got attached somewhere between the repo and your feed.

I am spending a paragraph on this because the reflex matters more than the file. Any accuracy claim about an agent that does not name a task set is a vibe, and our industry is currently drowning in them. Click the link before you install.

What is actually in there — and it is good

Strip the number away and four principles remain, and they are worth your time:

1. Think before coding — surface the silent assumption instead of guessing.
2. Simplicity first — the smallest thing that works.
3. Surgical changes — touch only what the task requires.
4. Goal-driven execution — define success criteria that can be verified.

The fourth one is the load-bearing wall, and the repo's own framing explains why: language models are exceptionally good at looping until they meet a specific goal. Give it criteria and it will grind toward them.

That maps to something I see constantly as an EM. Most bad output from Claude is not a capability failure — it is a specification failure. I told it what to do instead of what "done" looks like, and then I was surprised that it stopped somewhere I did not want.

Instructions vs. success criteria

The rewrite that changed the most for me:

# instruction — Claude decides when it's finished
Refactor the auth module to use the new session API.

# success criteria — the finish line is not mine to interpret
Refactor the auth module to the new session API.
Done means: every call site compiles, `pnpm test auth` exits 0,
and no file outside src/auth/ is modified.

Same task. The second one gives Claude something to check itself against, so it keeps going instead of handing you a half-migration that "looks right". It also gives you a fast way to know it is lying.

If you do turn it into a skill, know what a skill costs

The docs are clear about when a skill is the right container: create one when you keep pasting the same checklist into chat, or when a section of your CLAUDE.md has grown from a fact into a procedure. The economics are the point — a skill's body loads only when it is used, so long reference material costs almost nothing until you need it. That is the opposite of CLAUDE.md, which loads in full, every session.

The shape is minimal — .claude/skills/<name>/SKILL.md, with two frontmatter fields:

---
name: goal-check
description: Turn a vague task into verifiable success criteria before writing code
---

Before editing anything, restate the task as:
- One measurable end state (a test result, an exit code, a file count)
- The command that proves it
- What must NOT change on the way there

Then confirm the criteria with me before you start.

The description is not decoration — it is how Claude decides whether to load the skill at all. Write it as the trigger condition, not a summary. And note the directory name becomes the command (/goal-check), not the name field, for personal and project skills.

Test it on yourself instead of trusting the screenshot

Here is the honest version of what the viral post should have said: try it and measure. You do not need a research lab, just discipline.

1. Pick five real tasks from your own backlog — not toy problems.
2. Run them without the skill. Write down, for each: did it need a correction from you, and did the tests pass on the first attempt?
3. Install the skill. Run five comparable tasks the same way.
4. Compare the correction count.

That gives you a number about your repo, which is the only number that should change your setup. Mine went in the right direction — enough that I kept the goal-criteria habit and dropped the rest of the file. Your result may be different, and that is exactly the point.

Do this today

Take the last prompt you gave Claude Code that produced something almost-right. Rewrite it with a "Done means:" line containing a command and an exit code. Run it again. That comparison will teach you more than any percentage you did not measure yourself.

Sources

• multica-ai/andrej-karpathy-skills on GitHub
• Claude Code docs — Extend Claude with skills