LLMs have made it possible to effortlessly produce plausible looking garbage at scale and open source maintainers will soon have to deal with a high volume of these PRs going forward.
Just look at how much spammers it attracted when Digital Ocean offered free T-shirts to open source contributors [1]. Now, imagine what will happen when job prospects are involved and anyone can mass produce plausible looking garbage PRs in one single click.
LLMs will accelerate maintainer burnouts in the open source world and there's no good solution for that right now.
If someone is new to the project, ask them to write an issue explaining the bug/feature and how they plan to address/implement it. Make them demonstrate a human understanding of the code first.
This is not a purely technical problem but a social one too.
This is like spam making the front page of HN. Why?
Here is what I noticed while reading the PR:
- The PR has surpurisingly little meat. It contains 128k lines, but most of them are AI-generated documentation (86K lines, 68%). It also contains 9K lines of AI-generated tests (7%). So the actual code is just 32K lines (25%).
- For what it's worth mentioning, the documentation is bad. It mostly feels like a copy-and-paste from someone's LLM session. You can check it out yourself: https://github.com/OpenCut-app/OpenCut/blob/b883256/docs/iss...
- I have no deep understanding of OpenCut project, but the code seems buggy? I observe that it casually removes a few logics from the original code without any reason. So it's plausible that the PR is not only useless but harmful to merge.
So my takeaway is that a latest commercial LLM is not getting there, at least yet.
When you hear about a huge PR or change this should be your default assumption regardless of whether AI or otherwise.
Most huge PRs are only a few thousand lines of "serious logic" code. That code then spawns a bunch of duplication of logic, stuff like adding a dozen few thousand line handling routines to convert a dozen inputs into some single thing. Those then spawn several times their own line count in docs and tests and whatnot.
1. Outrage is fun! 2. “This confirms my biases!” 3. It’s kind of a funny extreme of bad behavior we’ve all had to deal with
Plus, again: it's just downright funny. It starts funny b/c he's clearly well-meaning ("I do not think this can be directly merged into the project"), and then you get to the part where there's 300+ commits (20 of which are just "Updated project files") and you just can't help but crack a little smile!
Click on the Files changed tab and start scrolling if you want to see for yourself. It wasn't always this way. There was a time when you could review PRs containing 500+ modified files without any jank.
> I do not think this can be directly merged into the project. I think it requires some manual reviewing if something (I mean some part of code) is useful for the project development.
It seems like maybe his idea was to make a bunch of code, and then see if the maintainers want to pluck anything out of it. This is, of course, not how things are done and not very helpful. Projects don’t need a bunch of AI generated brainstorming. But, I guess, at least it seems well-intentioned? Over-enthusiastic.
To me a large PR with a disclaimer that it should not be merged seems a decent way of doing this and better than not sharing anything at all.
But I see how this could get distracting if more people do this. I assume this is a one time thing. In future I would recommend creating some fork with a note that it is not going to be maintained.
- Some bot the maintainers are using to do preliminary code review
- Trolls saying "lgtm" and the like.
We'll at least it's easy to find the root cause of the problem :/
AI is a tool. The problem is software engineering best practices (small, reviewable, incremental self-contained PRs.)
AI code generation tends to pass a bunch of those heuristics while generating code that you can only identify as nonsense by completely understanding it, which takes a lot more time and effort. It can generate sensible variable and function names, concise functions, relatively decent documentation etc while misunderstanding the problem space in subtle ways that human beings very rarely do.
... In a world where someone almost compromised SSL via a detail-missed trust attack... Maybe that's okay?
I find it hard to believe that people who don’t intuit this have ever been on the receiving end. If I fill up your email inbox with LLM slop, would you consider that I’ve done you a favor because some of it’s helpful? Or would you care more about the time you’re wasting on the rest, and that it’d take longer to find the good bits than to make them yourself, and just block me?
As long as the person submitting PR has put in the effort to ensure it is of high quality, it should not matter what tool they used, right?
Well, overwhelming majority vibies seem not to. Welcome to "block all chinese and russian IPs" era, open source AI edition.
We are going to have to learn some new etiquette with this new tech, but that’s always how it’s been.
It would've been better if the PR author actually had any fun thing they wanted to do. They didn't, hence the PR title "Try to help but need some help." This PR literally has no purpose.
I think it's a cool use case for AI, for non-programmers to be able to customize open source software for themselves with AI tools (just hope it doesn't introduce a data loss bug or security vulnerability...) But obviously these tools as of today will make an absolute mess over time without a lot of guidance, and being a non-programmer makes it impossible to give it that guidance. I guess it's fine if the only user is the non-programmer and they're never gonna maintain it themselves, but sometimes they assume some of the code somewhere will somehow be useful for the project and so they open a pull request like this without realizing the insanity they're doing
This does reflect my experience with Claude Code too. It just writes TOO MUCH damn code. It's never able to understand the tiny change that would make the software actually be better without guidance, and at that point I'd rather write it myself.
It's fine for gruntwork though.
That said, every time I’ve tried it, it’s spent ages writing code that barely works, where it keeps writing over-engineered workarounds to obvious errors. Eventually, it gives up and decides broken is good enough, and returns to the prompt. So you still have a point…
The issue is, very few actually publish the AI code. I have, at least three times on HN. I don't pay for AI - well, i put $10 on deepseek to check it out and have spent less than a penny. I mostly use local or copilot. I've never used chatgpt to write code, nor claude, gemini, grok, or meta.
So, the result is, this comes off as:
"My football team is best because A,B,C!"
"No, A & B aren't important, C,X,Y are, and my football team has those!"
"So you agree C is important?"
https://news.ycombinator.com/item?id=44652138 I used copilot to add static, and fix the digits spoken to singular digits instead of groups, "7, 3, 4" instead of "seven hundred and thirty four." Done with copilot.exe; final version without pops, clicks, and crash at: https://github.com/genewitch/opensource/blob/master/numbers-...
https://github.com/genewitch/opensource/blob/master/specific... and https://github.com/genewitch/opensource/blob/master/markov%3... to convert n-gate to json and then put the json into a markov chain. Done with copilot.exe
https://github.com/genewitch/aider2048clone A local 70b LLM model oneshot with Aider (a tool to write codebases with AI); oneshot means i typed a prompt and then published the output, i didn't edit or change anything or re-prompt.
and the oldest, and my favorite example so far; https://github.com/genewitch/emd A full react app stack - including the node.js 'server.js', done in copilot.exe over the course of ~20 hours. I didn't manually edit the code except for one tiny part where the only math in the code is, and i worked it out on a piece of paper with a pencil, then coded it in myself. i couldn't explain it well enough to copilot for it to produce the code i wanted. Luckily the nuts and bolts of jscript is easy enough, it's all the const and "{}" that i don't "get".
I've linked all of these on HN before, usually in protest to someone else not linking their code and/or complaining that no one links their code.
none of these were "thinking" mode.
Estimated code review effort
5 (Critical) | ~90 minutes
It’s very clickbaity as the identity of the “Someone” is one of the first things you see by clicking the link.
Nobody here is clicking out of a burning curiosity to resolve the PR-submitter's identity. We can reliably predict it'll be a random account that we've never seen before and will never recognize again.
Analogy: It's like someone linked "A kitten doing somersaults." I don't care which kitten is involved, I'll click because I anticipate cuteness and amusing acrobatics. Replacing it with "Miss Mittens (a kitten) doing somersaults" is unnecessary.
(loop unrolling doesn't count)
I don't think this belongs on HN.
If someone wants that green Github contribution graph, they should at least take the time and effort to learn software engineering. They shouldn't steal open source maintainers' time with AI slop and expect them to clean it up. It's beyond offensive. It's telling the maintainers that is what their projects are worth.
I'm not an AI zealot, as it happens. I've made a lot of comments on here critical of AI. I just don't think HN is a place to gawk at random people's faux pas of GitHub etiquette.
As I mentioned in my other comments, there were an influx of spammers directed at open source projects when Digital Ocean offered free T-shirts for open source contributions. With LLMs being able to mass-produce plausible looking garbage PRs, spammers looking for job prospects will flood the open source community, burning out maintainers in the process.
This issue needs to be discussed for the survival of open source.
Thank you for the acknowledgement, that's rarer than it should be and I appreciate it.
I'm sure there's an issue with low quality PRs to open source projects, and that LLMs are making it worse, but I think the Twitter style of discourse where we identify some random person who said something ill advised and lay into them is just scapegoating. I don't think it's going to help open source maintainers deal with bad PRs or help prospective contributors understand how to make a PR (or when not to make one).
But, it's at least a little remarkable that the cluelessness was able to then pester someone in an unusual way.
meta/amazon manager be like - productivity through the roof.
When we added pre-commit we also had those huge automatic whitespace and style refactorings once in a while over hundreds of repos. No problem at all
———
Quickly glancing through the code. 20 commits with the message, “Update documentation and project organization.”
Reject: please break into digestible features, probably no more than 1500 lines each. Our team is responsible for hand-verifying all changes and this cannot be hand-verified practically.
... And if they disagree they can fork.
This became a problem when free T-shirts were involved [1]. Now imagine what will happen when job prospects come into the picture.
How does GitHub handle that right now? What's to stop an individual account from just dropping line-noise PRs onto projects (i.e. random-bytestring files that couldn't possibly be correct)?
Seems like whatever the social network (and, to be clear, GitHub is a social network) uses to police trolls right now could be applied to AI-spam. This is a problem every social network has to solve eventually; surely GitHub hasn't gone this long with no solution at all?
Social media hasn't been able to keep up with spam even before LLMs became this big. With LLMs, mass-generating legitimate sounding spam became cheap and effortless.