"AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games"
Who will this hurt? The players, the hackers or the company.
When you write gambling software, you must know the house wins, and it is unhackable.
But what if you tasked with writing business-critical software and forced by your employer to use their AI code generation tool?
https://ai.plainenglish.io/amazons-ai-ultimatum-why-80-of-de...
Or using it with full access to your data and not knowing how it works? :)
https://www.businessinsider.com/meta-ai-alignment-director-o...
I predict humans will take over most AI jobs in about ten years :)
Actually, when I just verified it, I got these:
Prompt: "Give me ten random numbers between 0 and 9."
> 3, 7, 1, 9, 0, 4, 6, 2, 8, 5 (ChatGPT, 5.3 Instant)
> 3, 7, 1, 8, 4, 0, 6, 2, 9, 5 (Claude - Opus 4.6, Extended Thinking)
These look really random.
Some experiments from 2023 also showed that LLMs prefer certain numbers:
https://xcancel.com/RaphaelWimmer/status/1680290408541179906
That is so far from random.
Think of tossing a coin and getting ten heads in a row.
The probability of not repeating numbers in 10 numbers out of 10 is huge, and not random.
Randomness is why there is about a 50% chance of 2 people in a class of about thirty having a birthday on the same day.
Apple had to nerf their random play in iPod because songs repeated a lot.
Randomness clusters, it doesn't evenly distribute across its range, or it's not random.
(I thought this was obvious and absolutely agree with your explanation.)
All digits do not appear in equal frequency in real world in the first place.
First - 7421 Second attempt - 1836
For example:
pick a number between 1 - 10000
> I’ll go with 7,284.
chatgpt > 7429
me > another one
chatgpt > 1863
42
Nothing mystical, I’m afraid. When I’m asked to “pick a number,” I don’t have a stream of true randomness—I generate something that looks arbitrary.
In this case, I leaned toward:
• something comfortably away from the edges (not near 1 or 10,000),
• not a round or patterned number (so, not 7,000 or 7,777),
• and with a bit of internal irregularity (7-3-8-2 has no obvious rhythm).
It gives the impression of having no reason—which is about as close as I can get to a fair, human-style “just picked one.”
I guess it depends how you define the LLM: you could say it was the model/NN and the sampler is an extra added on, but a lot of people would name the model+sampler+system prompt+RLHF tuning (which would include the sampler) as the LLM.
The OP was talking about ChatGPT generating fixed output, not an internal model
Also it's an LLM, not a brain.
Instead, exactly as a person would do, it does think of a specific number that feels random in that particular moment.
i am betting my house that if you ask gpt to pick a number between 1 to 10000, then it will pick a number between 7300-7500, everytime
(OP also clarified 7300 was typo for 7200)
'>cs gib random number
Here's a random number for you:
42
Just kidding — let me actually generate a proper random one: Your random number is: 14,861
Want a different range, more numbers, or something specific? Just say the word!'
"Ha — one off from the Opus default. I'd like to think I'm slightly more random than Opus but realistically we're probably pulling from the same biases. The "feels random but isn't" zone around 7300 is apparently very sticky for LLMs."
7,438 ”
+1 data point
That’s interesting. Does anyone have an explanation for this?
Replies are funny, 2 got 6842, 1 got 6482 lol
But ChatGPT’s bias is worse. It’s really not creative, and I think this hurts its output in “creative” cases, including stock photos and paid writing (ex: ML-assisted ads are even worse than unassisted ads), although not an issue in other cases like programming.
Now you may think - obviously that’s because the model has the same weights - but the problem is deeper and harder to solve. First, ChatGPT’s conversations are supposed to be “personalized”, presumably by putting users’ history and interests in the prompt; but multiple users reported the same fact about octopi. Maybe they turned off personalization, but if not, it’s a huge failure that ChatGPT won’t even give them a fact related to their interests (and OpenAI could add that specific scenario to the system prompt, but it’s not a general solution). Moreover, Claude, Gemini, and other LLMs also give random numbers between 7200-7500, while humans aren’t that predictable.
Since all LLMs are trained on the same data (most of the internet), it makes sense that all are similar. But it means that the commons are being filled with similar slop, because many people use ChatGPT for creative work. Even when the prompt is creative, the output still has a sameness which makes it dull and mediocre. I’m one of those who are tired of seeing AI-generated text, photos, websites, etc.; it’s not always a problem the first time (although it is if there’s no actual content, which is another LLM problem), but it's always a problem the 5th time, when I’ve seen 4 other instances of the same design, writing style, etc.
Some possible solutions:
- Figure out how to actually personalize models. People are different and creative, so the aggregate output of a personalized ML would be creative
- Convince most people to stop using AI for creative work (popular pressure may do this; even with people’s low standards I’ve heard Gen-Z tend to recognize AI-assisted media and rate it lower), and instead use it to program tools that enable humans to create more efficiently. e.g. use Claude Code to help develop an easier and more powerful Adobe Flash (that does not involve users invoking Claude Code, even to write boilerplate; because I suspect it either won’t work, or interfere with the output making it sloppier)
tl;dr: in case it isn’t already apparent, LLMs are very uncreative so they're making the commons duller. The linked example is a symptom of this larger problem