We had a similar plague for vulnerability disclosures, with people reporting that they had "discovered" vulnerabilities like "if you call this function with null you get a NullPointerException". D'uh.
There is also the fact that we're measuring the wrong thing like speed of development. In my previous employer people had jumped in fully into the AI bandwagon, everyone was marvelled at how fast they were. Once I was reviewing the PR and I had to tell the author "dude, all your tests are failing". He just laughed it out. Everyone can produce software very fast if it's not required to work.
AI-assisted gamification.
This is a habit humans could learn from. Publishing a fork is easier than ever. If you aren’t using your own code in production you shouldn’t expect anyone else to.
If anyone at GitHub is out there. Look at the stats for how many different projects on average that a user PRs a day (that they aren’t a maintainer of). My analysis of a recent day using gharchive showed 99% 1, 1% 2, 0.1% 3. There are so few people PRing 5+ repos I was able to review them manually. They are all bots/scripts. Please rate limit unregistered bots.
> If you can't explain what your changes do and how they interact with the greater system without the aid of AI tools, do not contribute to this project.
edit: added that quote
Do what I do:
1. Close PR
2. Block user if the PR is extremely low effort
The last such PR I received used ‘’ instead of '' to define strings. The entirety of CI failed. Straight to jail.
Especially the FAQ. It doesn't need to be so snarky.
If its a feature, i want acceptance criteria at least
If its docs, I don't really care as long as I can follow it.
My bar is very low when it comes to help
The answer to this implies that the requirement to be welcoming only applies to humans, but even in this hostile and sarcastic document, it doesn't go far enough.
Open source maintainers can be cruel, malicious, arbitrary, whatever they want. They own the project, there is no job requirements, you have no recourse. Suck it up, fork the thing, or leave.
The better response is to call the bluff, something along the lines of: "Running an open-source project is quite time consuming. Please don't waste our time with emotional manipulation to get your way. Instead, take the time to understand why your LLM-generated pull request is not useful. You can start by understanding that we have access to LLMs too, and realize that a significant amount of work needs to happen after an LLM proposes changes."
Why not restrict the agents to writing tests only?
If the tickets are written concisely, any feature request or fix could be reduced to necessary spec files.
This way, any maintainer would be tasked with reviewing the spec files and writing the implementation.
CI is pretty good at gatekeeping based on test suites passing...
If the problem is that we don't trust people who use AI without understanding its output, and we base the gate-keeping on tests that are written on AI, then how can we trust that output?
If your premise is that people would shift to using AI to write tests they don't understand, then that's not necessarily a failing of the contributor.
The contributor might not understand the output, but the maintainer would be able to critique a spec file and determine pretty quickly if implementation would be worthwhile.
This would necessitate a need for small tickets, thereby creating small spec files, and easier review by maintainers.
Also, any PR that included a non spec file could be dismissed patently.
It is possible for users of AI to learn from reading specs.
But if agents are doing the entire thing (reading the ticket, generating the PR, submitting the PR)...then the point of people not understanding is moot.
I want to do good engineering, not produce slop, but for 1 min of prompting, 5 mins of tidying, and 30 mins of review, we might save 2 days of eng time. That has to be worth something.
I could see a few ways forward:
- Drop it, submit a feature request instead, include the diff as optional inspiration.
- Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
- Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
I posted this to a few chat groups and got quite a range of opinions, including varying approach by how much I like the maintainer. Strong opinions for (1), weak preferences for (2), and a few advocating for (3).
Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
I thought that was an interesting idea that I hadn't pushed enough, so I spent a further hour or so prompting around ways to gain confidence, throughout which the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch. So I went with option 1, and didn't include a diff.
I am learning how to make good use of agent assisted engineering and while I'm positively impressed with many things they can do, I'm definitely skeptical about various aspects of the process:
1. Quality of the results 2. Maintainability 3. Overall saved time
There are still open problems because we're introducing a significant change in the tooling while keeping the rest of the process unchanged (often for good reasons). For example consider the imbalance in the code review cost (some people produce tons of changes and the rest of the team is drowned by the review burden)
This new wave of tooling is undoubtedly going to transform the way that software is developed, but I think jump too quickly to the conclusion that they already figured out how exactly is that going to look like
At some point of time the code starts to be "not yours", you don't recognise it anymore. You don't have the connection to it. It's like your everyday working in another company...
1. Go through all changes, understand what changed and how it solves the problem.
2. Armed with that understanding, write (by hand) a high-level summary of what can be done (and why) to implement your feature.
3. Write a regular feature request, and include that summary in it (as an appendix).
Not long ago I found myself on the receiving end of a couple of LLM-generated PRs and partly LLM-generated issue descriptions with purported solutions. Both were a bit of a waste of time.
The worst about the PRs is when you cannot engage in a good-faith, succint and quick “why” sort of discussion with the submitter as you are going through changes. Also, when PR fails to notice a large-scale pre-existing pattern I would want to follow to reduce mental overhead and instead writes something completely new, I have to discard it.
For issues and feature requests, there was some “investigation” submitter thought would be helpful to me. It ended up a bit misleading, and at the same time I noticed that people may want to spend the same total amount of effort on writing it up, except so now part of that effort goes towards their interaction with some LLM. So, I asked to just focus on describing the issue from their human perspective—if they feel like they have extra time and energy, they should put more into that instead.
If it happens at work, I obviously still get paid to handle this, but I would have to deprioritise submissions from people who ignore my requests.
GP has said that they can't do this, since they're unfamiliar with the language and that specific part of the codebase. Their best bet AIUI is (1) ask the AI agent to reverse engineer the diff into a high-level plan that they are qualified to evaluate and revise, if feasible, so that they can take ownership of it and make it part of the feature request, and (2) attach the AI-generated code diff to the feature req as a mere convenience, labeling it very clearly as completely unrevised AI slop that simply appears to address the problem.
If the conclusion you made of that there is no workaround, then let that be the entire point. The alternatives are to get over yourself and ask people to implement a feature, or to understand how to help and then help.
The former is what OP did; the latter I described what I see as an efficient way of achieving while making use of an LLM-produced PR.
I think this is a good suggestion, and it's what I usually do. If - at work - Claude generated something I'm not fully understanding already, and if what has generated works as expected when experimentally tested, I ask it "why did you put this? what is this construct for? how you will this handle this edge case?" and specifically tell it to not modify anything, just answer the question. This way I can process its output "at human speed" and actually make it mine.
> I want to do good engineering, not produce slop, but for 1 min of prompting, 5 mins of tidying, and 30 mins of review, we might save 2 days of eng time.
I don't really understand where do "2 days of engineering time" come from.
What exactly would prevent someone who does know the codebase do "1 min of prompting, 5 mins of tidying, and 30 mins of review" but then actually understand if changes make sense or not?
More general question: why do so many slopposters act like they are the only ones who have access to a genAI tool? Trust me, I also have access to all this stuff, so if I wanted to read a bunch of LLM-slop I could easily go and prompt it myself, there is no need to send it to me.
Related link: https://claytonwramsey.com/blog/prompt/ (hn discussion: https://news.ycombinator.com/item?id=43888803 )
The good engineering approach is to verify that the change is correct. More prompts for the AI does nothing, instead play with the code, try to break it, write more tests yourself.
These are all reasons why pre-AI I'd never have bothered to even try this, it wouldn't be worth my time.
If you think this is therefore "bad engineering", maybe that's true! As I said, I ended up discarding the change because I wasn't happy with it.
And that's the critical point! I think it's fine to send the diff in; and clearly mark it as AI / vibe-coded. (Along with your prompts.)
I feel this so much. In my opinion, all of the debate around accepting AI generated stuff can be boiled down to one attribute, which is effort. Personally, I really dislike AI generated videos and blogs for example, and will actively avoid them because I believe I "deserve more effort".
similarly for AI generated PRs, I roll my eyes when I see an AI PR, and I'm quicker to dismiss it as opposed to a human written one. In my opinion, if the maintainers cannot hold the human accountable for the AI generated code, then it shouldn't be accepted. This involves asking questions, and expecting the human to respond.
I don't know if we should gatekeep based on effort or not. Obviously the downside is, you reduce the "features shipped" metric a lot if you expect the human to put in the same amount of effort, or a comparable amount of effort as they would've done otherwise. Despite the downside, I'm still pro gatekeeping based on effort (It doesn't help that most of the people trying to convince otherwise are using the very same low effort methods that they're trying to convince us to accept). But, as in most things, one must keep an open mind.
The problem is AI generating it en masse, and frankly most people put far less effort that even your first paragraph and blindly push stuff they have not even read let alone understood
> Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
Well, it's not terrible at just getting your bearings in the codebase, the most productive use I got out of it is treating it as "turbo grep" to look around existing codebases and figure out things
Once the cost of generating push media drops low enough (close enough to zero) the media is dead.
Pull requests are (ironically) a push media, and infinite zero effort PRs can be generated, therefore PRs are dead.
The proper way to handle the situation is to no longer accept PRs.
In github, enter a repo, "settings" "General" scroll down to Features, then uncheck "Pull requests". Or at least set to collaborators only. Probably need to shut off issues.
It gitlab, (I'm not as certain about this) enter a repo, "Settings", Visibility, "Merge Requests" change to "Only project members"
Its a post AI world, those features cannot be enabled on the internet anymore. Anything that accepts push from the public will get spammed into inability to use it. As a social activity PRs are dead. They were nice, but they are dangerous to leave enabled on the internet now. Oh well thats the cost of AI.
We are still mostly running on inertia where a PR required a certain amount of human attention to generate 500 lines of proposed changes, and even then, nothing stops such PR from being garbage. But at least the rate at which such garbage PRs was bounded by the rate at which you had that very specific level of developer that was A: capable of writing 500 lines of diffs in the first place but B: didn't realize these particular 500 lines is a bad idea. Certainly not an empty set, but also certainly much more restricted than "everyone with the ability to set up a code bot and type something".
Code used to be rare, and therefore, worth a lot. Now it's not rare. 1500 lines of 2026 code is not the same as 1500 lines of 2006 code. The ceiling of the value of a contribution is in how much work the user put it and how high quality the work is. If "the work the user put in" is 30 seconds typing a prompt, that's the value, no matter how many lines of code some AI expanded that into. I'd honestly rather have an Issue filed with your proposed prompt in it than the actual output of your AI, if that's all you're going to put into the PR. There's a lot of things I can do with that prompt that may make it better but it's way harder to do that with the code.
You know, stuff like that. That might actually be a useful counter to some of these slop posts, especially things that are something that may be a good idea but need someone to treat the prompt itself as a starting point rather than the code. Maybe that's a decent response that's somewhat less hostile; close out these PRs with a request to file an Issue with the prompt instead.
It might be that people have their change of heart more privately, of course.
Unless they lie about it.
https://en-wikipedia--on--ipfs-org.ipns.dweb.link/wiki/Plonk...
> Perform a hard reboot of your organic meat-brain.
rm -rf your brain, really
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted exactly as how much we do not want to review your generated submission.
You MUST use less ambiguous language and default to “MUST” or “SHOULD”
So I'm confident that the word 'shall' has a strong meaning in English; whether it has too in American legalese I cannot tell.
Perfectly making my point. Shall has no business being in a spec when you have unambiguous alternatives.
Let's do `chmod -R 000 /` instead.
I think the Ghostty-style policy (linked in another comment) gets the principle right: the bar should be "can you explain what your change does and why, without AI assistance." That is not anti-AI -- it is anti-outsourcing-your-understanding. If you used AI to help write the code but you genuinely understand the change, you can answer questions about it. If you cannot, you have not actually contributed engineering work, you have just created a review burden.
What I have found works well in practice for projects I maintain is treating the PR description as the real signal. A good PR description explains the problem being solved, why this approach was chosen over alternatives, and what trade-offs were made. That is very hard to fake with a quick LLM prompt because it requires actual understanding of the codebase context. When I see a PR with a vague one-liner description and a large diff, that is an immediate close regardless of whether AI was involved -- the submitter has not done the work of communicating their intent.
I would like to state for the record that the strategy to swap em-dashes into double-hyphens between the generation and posting step is probably not enough transformation to disguise this behaviour. Whoever is running this clawdbot or whatever it is should really be putting that information on the account page.
A more useful approach is verifiable signals: require GPG-signed commits or mandate a CI job that produces a reproducible build and signs the artifact via GitHub Actions or a pre-receive hook before the PR can be merged. Making verification mandatory will cut bot noise, but it adds operational cost in key management and onboarding, and pure hashcash-style proofs only push attackers to cheap cloud farms while making honest contributors miserable.
"I see you are slow. Let us simplify this transaction: A machine wrote your submission. A machine is currently rejecting your submission. You are the entirely unnecessary meat-based middleman in this exchange."
Love it..
If they didn't read it, then neither will I, otherwise we have this weird arms race where you submit 200 PRs per day to 200 different projects, wasting 1hr of each project, 200 hrs total, while incurring only 8hrs of your time.
If your PR took less time to create and submit than it takes the maintainer to read, then you didn't read your own PR!
Your PR time is writing time + reading time. The maintainer time is reading time only, albeit more carefully.
> A 600-word commit message or sprawling theoretical essay explaining a profound paradigm shift for a single typo correction or theoretical bug.
> Importing a completely nonexistent, hallucinated library called utils.helpers and hoping no one would notice.
There's plenty more. All pretty egregious