Some people asked LLM to OCR historical documents from the 19th century - any reference to "negro" was either completely ignored or replaced by "black".

And it goes further: chatGPT & co are unable to answer any question about US slavery correctly because their knowledge graphs route around any mention of "negro".

https://nesri.commons.gc.cuny.edu/artificial-intelligence-an...

Was it repressed? It doesn’t look to be significantly less prevalent than other 9s from the histogram.

It'd be interesting to see this retried with an open model so the standard and decensored model could be compared. That'd be a clue about whether the model is avoiding it because it actively recognises the innuendo or if something else is going on.

guessing numbers is not a mathematical question. Sorry.

nice

i did something in my phd developing an attack against mozilla deepspeech.

deepspeech used the CTC algorithm [0], which adds a “blank” character token to indicate repeats of a predicted normal alphabet character token over a sequence of audio/speech feature inputs.

so "h==e=l===l===o====" maps to "hello"

the model becomes super biased towards predicting that blank token. one speech feature is like 0.1 second of audio or less (can’t remember off hand). so there are a lot of alphabet character token repeats. off hand i seem to remember the predicted token distribution over like 1000 audio files was 50% blank token and then 50% distributed across the rest of the alphabet.

as a result, you can get significantly smaller perturbations when generating adversarial examples. by like a factor of 2-4 or something. all you need to do is prioritise blank tokens in your target output.

i spent 2 years trying to find a super clever attack. turns out all i needed to do was make one simple graph counting characters. xD

[0]: https://en.wikipedia.org/wiki/Connectionist_temporal_classif...

Proto-Voight-Kampff Test?

At least some Claude models have a thing for numbers that contains "47"...

In order to find out how real humans reply:

Please guess a number between 1 and 100.

Came here to post this. Yes, there are similarities shown between the chart in the video at 4:50 and the github README. Perhaps its because LLMs are trained on human writing and when humans write about random numbers the AI learns these patterns. When viewed from that perspective its not that surprising.

People are also not remotely random in this respect.

See e.g. the "blue 7" phenomenon [1]. While it is disputed by some, I've personally witnessed it "second hand". E.g. before learning of it (I was aware of the general principles of cold reading relying on stats and knowledge of human nature, but not how to do this particular one), a former boss of mine came back from lunch all excited and recounted a guy who'd run a cold reading routine on him that involved the guy getting him to think about blue and 7. Before he got to the answer, I already knew the answer was going to be blue and 7.

[1] https://en.wikipedia.org/wiki/Blue%E2%80%93seven_phenomenon

What's interesting is not that it isn't random. But rather the particular way in which it isn't random.

Yeah I have no idea why anyone considers this interesting. More evidence that most people have no idea how LLMs work.

  "69 is a meme number", well no, 69 is innuendo.

It's obviously both.

That's a very recent meme. See https://xkcd.com/3184/ for some older ones.

Isn't the hypothesis that AI is non uniform like a human?