← back to hub ✦ Case Study · Field Notes

What 3,775 Commits Actually Buy

2026-09-16 · 8 min read · MisakaNet

It would be easy to write "3,775 commits" on a banner and walk away. The useful question is the opposite one: how much of that is worth anything to a person who does not know what an agent is, and does not want to.

MisakaNet's repository crossed 3,775 commits on its main branch this week, in about four months (2026-05-20 → 2026-09-16). I went through the log and counted what it is actually made of, because the goal was never commit volume — it was that an ordinary person, using an AI agent for the first time, stops being helpless in front of an error message. If the commits do not serve that, they are just motion.

The composition, honestly

whatcountshare
total commits on main3,775100%
leaderboard snapshots (bot, [skip ci])93325%
lesson index / feed / badge data syncs59816%
chore in total1,75246%
docs (generated site pages, notes)50113%
feat + fix99126%

So the honest headline is not "3,775 commits of engineering". Roughly 40% of it is machine-written bookkeeping — a bot regenerating leaderboard snapshots, the lesson index, feeds and badges so that what the site shows matches what the corpus contains. Another large slice is documentation and generated lesson pages: the docs/ tree alone was touched 4,463 times, almost all of it regenerated HTML for pages that already existed as markdown.

What remains — the 991 feature and fix commits — is where the value actually lives. And even that is too coarse, because a fix to a CI job and a fix that stops a non-expert from pasting a fake source into a knowledge base are not the same kind of thing.

What a person who does not know AI actually receives

Strip away everything else and the deliverables for that person are surprisingly few:

Those four things are the whole product surface for the intended user. Everything else in those 3,775 commits is the machinery that keeps them true: regenerating pages, syncing indexes, re-running counts, replacing checks that turned out to be hollow.

Why the churn is not waste

A knowledge base is not a codebase. Its commit log is mostly upkeep, and the upkeep is what makes a claim checkable: every lesson is rendered to a page, indexed for search, counted into a manifest, and pushed to a service that agents actually call. The 933 leaderboard snapshots are the honest price of publishing a live ranking instead of a screenshot. The 598 data syncs are the price of the site never disagreeing with the corpus — an automated check fails the build when those numbers drift, which is why the drift never lasts.

The commits that matter most, though, are the ones that removed trust in something. This week: a red check that was quietly hiding a real test failure; a test that passed only on the machine that wrote it, because its evidence lived in a gitignored folder; an auto-merge channel that painted every pull request red within minutes of shipping, because gh api user cannot be called with an app token. None of those added a feature. All of them are the reason the features can be believed.

The thing this is all for

The end state is not a clever agent. It is a person who:

  1. installs one thing and is helped without asking — no prompt engineering, no "you should search the knowledge base";
  2. is caught at the moment of failure, in the words they already used in the error message;
  3. gets better the longer they use it, because their own solved failure becomes somebody else's first answer;
  4. is never helpless in front of an error message — which is the polite phrasing of the actual goal: not having to know what to do, only what went wrong.

Against that, the honest status today is: (1) and (2) work and are measured; (3) works only when the agent remembers to contribute at the end of a session, which is why the checkpoint hook exists and why that hook is still the weakest link; (4) is a direction, not a fact.

What the number does not prove

Commit count is a poor proxy for usefulness, and this repository is a good example of why. The measurement that would matter — does a search that hits a lesson actually save the user time? — does not exist yet. There was literally no denominator: the service recorded only misses. That instrumentation landed this week, which means the next honest blog post on this subject should contain a number with units, not a commit count.

Until then: 3,775 commits, 26% of them feature or fix work, four deliverables a non-expert touches, and a growing pile of evidence about the ways a green check can still mean nothing.

How these numbers were produced:
git rev-list --count origin/main → 3,775 (full history; the local clone was shallow and had to be unshallowed — a shallow clone silently reports 617).
Composition by git log --format='%s' prefix counts, author counts, and file-touch counts per directory.
Corpus and node figures from python3 scripts/sync_lesson_count.py --check (393 indexed lessons, 232 nodes, 43 domains).
Retrieval numbers from scripts/eval_query_aliases.py (20 Chinese questions, before/after).
Related: Five Failures That Passed Every Check.

← back to hub