AI Tells: the burden of LLMs

In July, the members of Codeberg voted to prohibit vibe-coded projects in their terms of use, on the grounds that these tools are incompatible with maintaining free software. Their reasoning is mostly about infrastructure — crawlers, hardware prices, donation money spent on machines — and it is theirs to make; I am not here to relitigate it. What it told me is that the question has stopped being rhetorical. A forge with real maintainers took a vote on whether this stuff is a net cost to them.

Which made me curious about a surface neither side of that argument tends to cite, because it happens to be public and I happen to have a tool for it: the writing. Every issue and pull request on GitHub is prose someone produced and someone else has to read. If these tools changed what arrives in a maintainer’s inbox, the text should show it.

I build Vale, a linter for prose, and someone has already done the hard part of naming the register: vale-ai-tells collects the vocabulary models overuse into a Vale style package, drawn from academic work on AI fingerprints and from Wikipedia’s own AI-detection guide. That makes a workable instrument. Point it at prose written before these tools existed, point it at prose written after, and see whether the numbers move.

So I did, across 2020 to 2025, on the least literary corpus I could think of: 26,658 opened issues from GH Archive, stripped of code fences and logs, scored against 176 flagged words. Bug reports are a hostile place to look for style. That is the point. Whatever turns up here is not someone trying to write beautifully.

The load is real, and it lands on pull requests

Start with the crudest version of burden: how much arrives. The windows below are different lengths, so everything is a per-month rate, and the pull-request counts drop Dependabot and the other familiar bots — a project switching on automated updates should not read as a surge in human contribution.

Show the data

Project	Change (%)
VS Code	191
Home Assistant	49
PyTorch	36
Rust	33
Go	9
Kubernetes	-27

Source: GitHub search API; collected in scripts/fetch-project-tells.mjs

Pull requests are up in 5 of 6 popular projects, and the leader is not subtle: VS Code went from about 289 human-opened pull requests a month to 841, a rise of 191%. That is the editor most of these assistants ship inside, now fielding roughly three times the patches it used to. I checked whether that was simply bots being miscounted. Excluding them barely moves it.

Issues are the weaker signal — up in only 4 of 6, and modestly where they rise. The asymmetry is the most useful thing in this piece. These tools lower the cost of submitting a change far more than the cost of noticing a problem, which fits what they actually do: a model can produce a diff on request, but it cannot observe that your thermostat integration stopped reporting overnight. The burden arriving in open source is review burden, specifically.

And the exception matters as much as the rises. Kubernetes is down 27% on pull requests and down on issues too, which is a reminder that project lifecycle swamps everything else in a volume number. A mature infrastructure project past its hype curve goes quiet regardless of what the tooling does. Volume measures a project’s whole situation at once, and anyone reporting it as an AI metric without saying so is selling you something.

The prose changed too

• Issues
• Pull requests

Show the data

Year	Issues	Pull requests
2020	0.328	0.218
2021	0.321	0.291
2022	0.368	0.208
2023	0.341	0.254
2024	0.514	0.247
2025	0.828	0.586

Source: GH Archive; scored in scripts/fetch-github-ai-tells.mjs

For four years the rate sits flat near 0.34 matches per thousand words. Then it climbs: 0.514 in 2024, 0.828 in 2025 — roughly 2.4× the pre-assistant baseline, with a 1.6× jump in the final step alone. Pull requests move the same way at a lower level, which fits, because a PR body is half template and a template does not change its diction.

Hourly archive files vary enormously in size, so a finding that only appears in the hours I happened to sample is not a finding. I re-scored the last step on hours the series never touched — same month, same hour of day, a year apart — and the rate went from 0.412 to 0.971, a 2.4× step. Different sample, same direction, similar size. The shift is real.

Length moved with it. Mean issue length rose from 74 words to 127, up 72%, and the replication hours agree (78 to 101). Bug reports are not becoming that much more informative on their own.

• Issues
• Pull requests

Show the data

Year	Issues	Pull requests
2020	74	46
2021	76	48
2022	83	68
2023	77	75
2024	81	93
2025	127	65

Source: GH Archive; scored in scripts/fetch-github-ai-tells.mjs

Not the words you were promised

Show the data

Word	Growth (x)
innovative	16.3
seamless	8.5
streamline	8.1
comprehensive	5.8
robust	4.7
ignite	4
evolving	3.8
navigates	2.8
actionable	2.3
scalability	2.2

Source: GH Archive; scored in scripts/fetch-github-ai-tells.mjs

The celebrity tell is delve, and delve is nowhere on this list. Nobody delves into a stack trace. What rose is the register of corporate competence: innovative, seamless, streamline, comprehensive, robust, actionable, scalability. Words that promise diligence.

Which makes sense once you picture the prompt. A model asked to file a bug report does not reach for lyricism; it reaches for the vocabulary of professional thoroughness. The tells are domain-shaped, and a detector tuned on essays will look straight past them. If you want to catch this register in a tracker, you want the commit-and-issue half of that package, not the general one.

Where it lands

Show the data

Project	Issues flagged (%)
PyTorch	3.91
Go	0.89
Kubernetes	0.78
VS Code	0.58
Rust	0.29
Home Assistant	0.23

Source: GitHub search API; collected in scripts/fetch-project-tells.mjs

Averages hide the thing you want to know, so I asked a narrower question of the same 6 projects: what share of a project’s issues contain at least one of the 5 words that rose most. Every one of them went up. Not a single project held flat, which is the strongest evidence here that something general is happening rather than one community drifting. But the levels differ wildly — PyTorch sits at 3.91% of issues, about 17× the 0.23% in Home Assistant.

Show the data

Project	Growth (x)
PyTorch	5.4
VS Code	4.5
Kubernetes	3.3
Go	2.7
Home Assistant	2.3
Rust	2.2

Source: GitHub search API; collected in scripts/fetch-project-tells.mjs

Growth runs 2.2× to 5.4×, and PyTorch both starts highest and climbs fastest — 0.72% to 3.91%, a 5.4× move. There is an irony in the framework most of this tooling is built with hosting the most machine-inflected prose in the sample, though I should immediately complicate it: robust and comprehensive are ordinary machine-learning vocabulary. An issue about robust training is using the word correctly, and some of that project’s level — and some of its growth, as the field moved toward exactly those topics — is subject drift rather than style. The probe cannot separate the two.

The bottom of the range is easier to read, and it is the practical finding. Home Assistant and Rust changed least, and both run strict issue forms: pick a category, fill the fields, paste the version string. A template is a constraint on prose, and a constrained field is a poor place to be expansive. If there is a practical defence in this data, it is that one: not detecting the register after the fact, but leaving less room to write it in.

The timing indicts the editor, not the chatbot

• OpenAI SDK
• Anthropic SDK

Show the data

Month	OpenAI SDK	Anthropic SDK
Jun 22	0M	0M
Jul 22	0M	0M
Aug 22	0M	0M
Sep 22	0M	0M
Oct 22	0M	0M
Nov 22	0M	0M
Dec 22	0M	0M
Jan 23	1M	0M
Feb 23	1M	0M
Mar 23	2M	0M
Apr 23	2M	0M
May 23	3M	0M
Jun 23	3M	0M
Jul 23	4M	1M
Aug 23	5M	1M
Sep 23	5M	1M
Oct 23	5M	1M
Nov 23	5M	1M
Dec 23	5M	1M
Jan 24	6M	1M
Feb 24	6M	1M
Mar 24	7M	2M
Apr 24	7M	2M
May 24	8M	3M
Jun 24	8M	2M
Jul 24	9M	2M
Aug 24	10M	2M
Sep 24	9M	2M
Oct 24	10M	3M
Nov 24	10M	3M
Dec 24	9M	3M
Jan 25	11M	3M
Feb 25	13M	4M
Mar 25	15M	5M
Apr 25	16M	5M
May 25	18M	6M
Jun 25	20M	7M
Jul 25	23M	8M
Aug 25	27M	9M
Sep 25	29M	10M
Oct 25	33M	11M
Nov 25	32M	12M
Dec 25	34M	13M
Jan 26	42M	17M
Feb 26	50M	24M
Mar 26	71M	41M
Apr 26	79M	64M
May 26	93M	85M

Source: npm registry; collected in scripts/fetch-ai-adoption.mjs

Here the easy story breaks. ChatGPT launched at the end of 2022, and if that were the cause, 2023 would show the break. It does not — 2023 is flat. The move belongs to the last two years, which is when assistants stopped being a website you pasted into and became something resident in the editor and the pull request: Copilot Chat, Claude, Cursor, agents that open patches themselves.

App installs are private, but developer uptake of the client libraries is not, and it is the same adoption curve from the supply side. Both bend upward through exactly the window where the prose moves, steepening hard in the last year. OpenAI’s package runs about 93.5M downloads a month and Anthropic’s 85.4M — roughly 91% of it, near parity, from nothing in 2023. On Python the gap is wider (398.6M against 185.1M, or 46%). None of this identifies who wrote any given sentence. It does say that the tools which write code arrived on the prose shift’s schedule, and the chatbot did not.

The tell that wasn’t

I expected the em-dash to carry this piece. It is the most-repeated fingerprint in every thread about spotting machine writing, and it is trivial to count.

It did not survive. In the sampled hours the rate rose; in the replication hours it fell, 0.613 to 0.391 per thousand words. Worse, the absolute rate differs by nearly an order of magnitude between two samples of the same year (0.129 against 0.391) — a few long issues by people fond of punctuation are enough to swing it. At this sample size the em-dash measures nothing at all.

That is worth stating plainly, because the claim circulates as though it were settled. On this corpus the boring vocabulary signal replicated and the famous punctuation signal did not. If you are judging a contribution by its dashes, you are not detecting a model. You are detecting a writer.

I ran it on myself

The obvious objection to all of this is that I am the sort of person who writes like the lexicon. Formal register, linter author, and those are words I use. So I pointed the same instrument at 1,487 of my own GitHub replies — 55,395 words across the projects I maintain, quoted text stripped, because that is someone else’s writing.

• My replies
• GitHub issues

Show the data

Year	My replies	GitHub issues
2017	1.239	—
2018	1.282	—
2019	0.369	—
2020	0.247	0.328
2021	0.178	0.321
2022	0	0.368
2023	0.2	0.341
2024	0.191	0.514
2025	0.581	0.828

Source: GitHub API and GH Archive; scored in scripts/fetch-my-tells.mjs

The objection lands, but not in the direction I expected. My rate is 0.379 per thousand overall — around GitHub’s pre-assistant baseline, well under its 2025 level — and the shape is backwards. My worst years were 2017 and 2018, peaking at 1.282, higher than anything GitHub has posted since, and five years before a model could write a paragraph. By 2025 I was at 0.581. The lines cross.

So I did write like an AI, in 2017, before there was an AI to write like. Which is the least mystical possible explanation of this whole register: the models did not invent it. They absorbed it from people who already wrote that way — some of us then drifted off it as the average drifted onto it. My particular tells are notably and aforementioned, the stiff connectives of someone recently out of a maths degree. What rose across GitHub was innovative and seamless, the vocabulary of product marketing. Same lexicon, opposite halves.

• My replies
• GitHub issues

Show the data

Year	My replies	GitHub issues
2017	49	—
2018	46	—
2019	48	—
2020	43	74
2021	33	76
2022	25	83
2023	29	77
2024	31	81
2025	39	127

Source: GitHub API and GH Archive; scored in scripts/fetch-my-tells.mjs

Length runs the other way too. GitHub’s issues grew; my replies shrank, from 49 words to about 39. That is less a style choice than what a decade of maintaining something does to a person. The prose you write at hour one of a project and at issue nine hundred are different prose. One caution I would raise about anyone else’s version of this chart: 21 flagged words in 55,395 is thin for a per-year line, so read the direction and the level, not the wiggles.

Commits are the other half, and the half that package has a dedicated ruleset for: agents are said to narrate themselves, justify at the end, and reach for gitmoji far more than people ever did. Git needs no API, so this one I could simply measure — 5,211 commits across 58 public repositories. The lexicon found 3 flagged words in 36,258 words (showcase, harness, granular, once each), a rate of 0.083 per thousand, an order of magnitude under what GitHub’s issues run at now. Gitmoji: 12 commits out of 5,211.

The subject line is the flat part — across 11 years it has stayed between 3.4 and 5.5 words, which is what the most constrained writing a programmer does looks like. What changed is underneath. This year my commits average 34.4 words against 4.7 before, with 21.4% carrying a real body, because I started writing the reasoning down: what broke, which path produced it, why the fix has the shape it has. The flagged rate for that year is 0.16. Seven times the prose, none of the register.

These are public commits only. Private work is invisible to this method, so the count is a floor rather than a total.

What a word count can tell you is narrow, and worth having anyway. Something did change, in 2024 and 2025 rather than at the chatbot’s launch. It arrives as review load rather than triage load. It is legible in a vocabulary of promised diligence rather than in the punctuation everyone polices. And the projects that felt it least were the ones that had already decided how much room a contributor gets to write in — which is the only finding here that suggests a lever.

The part I keep turning over is smaller than any of that. The register now read as machine-written is one humans built; the models learned it from us, and I was writing it in 2017. Whatever is arriving in these trackers, it is speaking in a voice we taught it. The tell was ours first.