Using secondary school maths to demystify AI
90 points
by zdw
7 hours ago
| 6 comments
| raspberrypi.org
| HN
Kim_Bruning
1 hour ago
[-]
Provocative title with a much more reasoned lede.

I'm pretty sure a set of workshops isn't ACTUALLY going to solve a problem that philosophers have been at each other's throats for for the past half century.

But BOY does it get people talking!

Both sides of the debate have capital-O Opinions, and how else did you want to drum up interest for a set of mathematics workshops. O:-)

reply
dang
42 minutes ago
[-]
Yes, that word 'think' is a hell of a distraction. We've replaced that bit of the title with a less provocative / more interesting phrase from the article.

Hopefully we can talk about the actual math and stuff (although the article doesn't go into much of that).

reply
t23414321
1 minute ago
[-]
artificial intelligence in place of artificial knowledge is confusing distraction too
reply
TallGuyShort
5 hours ago
[-]
It's unfortunate that there's so little (none in the article, just 1 comment here as of this writing) mention of the Turing Test. The whole premise of the paper that introduced that was that "do machines think" is such a hard question to define that you have to frame the question differently. And it's ironic that we seem to talk about the Turing Test less than ever now that systems almost everyone can access can arguably pass it now.
reply
fabianhjr
7 minutes ago
[-]
> “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.” ~ Edsger W. Dijkstra

The point of the Turing Test is that if there is no extrinsic difference between a human and a machine the intrinsic difference is moot for practical purposes. That is not an argument to whether a machine (with linear algebra, machine learning, large language models, or any other method) can think or what constitutes thinking or consciousness.

The Chinese Room thought experiment is a compliment on the intrinsic side of the comparison: https://en.wikipedia.org/wiki/Chinese_room

reply
bondarchuk
2 hours ago
[-]
I wish I would've learned about ANNs in elementary school. It looks like a worthwhile and cool lesson package, if only they'd do away with the idiotic dogma...
reply
astrange
1 hour ago
[-]
I took an AI class in 2010 and even then was mostly taught that neural networks aren't very useful and you should use random forests and MCMC.
reply
croemer
6 hours ago
[-]
Don't think this is very good - more of a report of their activities. Underdelivers on the headline.
reply
dang
40 minutes ago
[-]
[stub for offtopicness]

(in this case, thinkiness)

reply
causal
6 hours ago
[-]
I love the idea of educating students on the math behind AI to demystify them. But I think it's a little weird to assert "AI is not magic and AI systems do not think. It’s just maths." Equivalent statements could be made about how human brains are not magic, just biology - yet I think we still think.
reply
qsort
5 hours ago
[-]
I agree saying "they don't think" and leaving it at that isn't particularly useful or insightful, it's like saying "submarines don't swim" and refusing to elaborate further. It can be useful if you extend it to "they don't think like you do". Concepts like finite context windows, or the fact that the model is "frozen" and stateless, or the idea that you can transfer conversations between models are trivial if you know a bit about how LLMs work, but extremely baffling otherwise.
reply
gowld
3 hours ago
[-]
> Concepts like

> finite context windows

like a human has

> or the fact that the model is "frozen" and stateless,

much like a human adult. Models get updated at a slower frequency than humans. AI systems have access to fetch new information and store it for context.

> or the idea that you can transfer conversations between models are trivial

because computers are better-organized than humanity.

reply
isoprophlex
3 hours ago
[-]
> much like a human adult.

I do hope you're able to remember what you had for lunch without incessantly repeating it to keep it in your context window

reply
hatthew
57 minutes ago
[-]
My context window is about a day. I can remember what I had for lunch today, and sometimes what I had for lunch yesterday. Beyond that, my lunches are gone from my context window and are only in my training data. I have vague ideas about what dishes I ate, but don't remember what days specifically. If I had to tell you what separate dishes I ate in the same meal, I don't have specific memories of that. I remember I ate fried plantains, and I ate beans & rice. I assume they were on the same day because they are from the same cuisine, and am confident enough that I would bet money on it, but I don't know for certain.

One of my earliest memories is of painting a ceramic mug when I was about 3 years old. The only reason I remember it is because every now and then I think about what my earliest memory is, and then I refresh my memory of it. I used to remember a few other things from when I was slightly older, but no longer do, because I haven't had reasons to think of them.

I don't think humans have specific black and white differences between types of knowledge that way LLMs do, but there is definitely a lot of behavior that is similar to context window vs training data (and a gradient in between). We remember recent things a lot better than less recent things. The quantity of stuff we can remember in our "working memory" is approximately finite. If you try to hold a complex thought in your mind, you can probably do that indefinitely, but if you then try to hold a second equally complex thought as well, you'll often lose the details of the first thought and need to reread or rederive those details.

reply
mewpmewp2
2 hours ago
[-]
Wouldn't context be comparable to human short term memory, which could be neurons firing in a certain pattern repeatedly to keep it there?

How would you say human short term memory works if not by repeated firing (similar to repeatedly putting same tokens in over and over)?

reply
whoknowsidont
2 hours ago
[-]
A lot of people genuinely can't remember what they did an hour ago, but to be very clear you're implying that an LLM can't "remember" something from an hour, or three hours ago, when it's the opposite.

I can restart a conversation with an LLM 15 days later and the state is exactly as it was.

Can't do that with a human.

The idea that humans have a longer, more stable context window than LLM's, CAN or is even LIKELY to be true given certain activities but please let's be honest about this.

If you talk to someone for an hour about a technical conversation I would guesstimate that 90% of humans would immediately start to lose track of details in about 10 minutes. So they write things down, or they mentally repeat things to themselves they know or have recognized they keep forgetting.

I know this because it's happened continually in tech companies decade after decade.

LLM's have already passed the Turing test. They continue to pass it. They fool and outsmart people day after day.

I'm no fan of the hype AI is receiving, especially around overstating its impact in technical domains, but pretending that LLM's can't or don't consistently perform better than most human adults on a variety of different activities is complete non-sense.

reply
sincerely
53 minutes ago
[-]
The Turing test was passed in the 80s by Eliza it doesnt mean anything
reply
whoknowsidont
45 minutes ago
[-]
Why doesn't it mean anything?
reply
NooneAtAll3
2 hours ago
[-]
I do hope you're able to remember what buttons you just pressed without staring at your hands while doing so to keep it in your working memory

I do hope you're able to remember what was your browser tab 5 tab switches ago without keeping track of it...

reply
ux266478
5 hours ago
[-]
It's just provencial nonsense, there's no sound reasoning to it. Reductionism being taken and used as a form of refutation is a pretty common cargo culting behavior I've found.

Overwhelmingly, I just don't think the majority of human beings have the mental toolset to work with ambiguous philosophical contexts. They'll still try though, and what you get out of that is a 4th order baudrillardian simulation of reason.

reply
bruntofsaurus
2 hours ago
[-]
"Just" is used here as a reductive device. You reduce others to a few sentences.

Sentences constructed of words and representations of ideas defined long before you existed. I question whether you can work with ambiguous contexts as you have had the privilege of them being laid out in language for you already by the time you were born.

From my reference frame you appear to merely be circumlocuting from memory, and become the argument you make about others.

reply
jvanderbot
5 hours ago
[-]
Thinking is undefined so all statements about it are unverifiable.
reply
ben_w
4 hours ago
[-]
I would say a different problem:

There's many definitions of "thinking".

AI and brains can do some, AI and brains definitely provably cannot do others, some others are untestable at present, and nobody really knows enough about what human brains do to be able to tell if or when some existing or future AI can do whatever is needed for the stuff we find special about ourselves.

A lot of people use different definitions, and respond to anyone pointing this out by denying the issue and claiming their own definition is the only sensible one and "obviously" everyone else (who isn't a weird pedant) uses it.

reply
jvanderbot
4 hours ago
[-]
This is not a meta-question.

The definition of "thinking" in any of the parent comments or TFA is actually not defined. Like literally no statements are made about what is being tested.

So, if we had that we could actually discuss it. Otherwise it's just opinions about what a person believes thinking is, combined with what LLMs are doing + what the person believes they themselves do + what they believe others do. It's entirely subjective with very low SNR b/c of those confounding factors.

reply
BobaFloutist
4 hours ago
[-]
What's a definition of thinking that brains definitely provably can't do?
reply
ben_w
1 hour ago
[-]
Halting problem.

There are people who insist that the halting problem "proves" that machines will never be able to think. That this means they don't understand the difference between writing down (or generating a proof of) the halting problem and the implications of the halting problem, does not stop them from using it.

reply
_alternator_
2 hours ago
[-]
Computing the Kolmorgorov constant?
reply
BobaFloutist
1 hour ago
[-]
I don't know that I agree that computation is a variety of thinking. It's certainly influenced by thinking, but I think of thinking as more the thing you do before, after, and in-between the computation, not the actual computation itself.
reply
terminalshort
5 hours ago
[-]
Statements like "it is bound by the laws of physics" are not "verifiable" by your definition, and yet we safely assume it is true of everything. Everything except the human brain, that is, for which wild speculation that it may be supernatural is seemingly considered rational discussion so long as it satisfies people's needs to believe that they are somehow special in the universe.
reply
jvanderbot
5 hours ago
[-]
True. You need to define "it" before you can verify physics bounds it.

Unicorns are not bound by the laws of physics - because they do not exist.

reply
cwmoore
1 hour ago
[-]
They are, apparently, proscribed by the totality of the laws of physics. For now.
reply
wizzwizz4
1 hour ago
[-]
But every unicorn is bound by the laws of physics.
reply
gowld
3 hours ago
[-]
> it satisfies people's needs to believe that they are somehow special in the universe.

Is it only humans that have this need? That makes the need special, so humans are special in the universe.

reply
terminalshort
2 hours ago
[-]
It is bound by the same laws of physics as everything else, so no, not special.
reply
sublinear
1 hour ago
[-]
I think what many are saying is that of all the things we know best, it's going to be the machines we build and their underlying principles.

We don't fully understand how brains work, but we know brains don't function like a computer. Why would a computer be assumed to function like a brain in any way, even in part, without evidence and just hopes based on marketing? And I don't just mean consumer marketing, but marketing within academia as well. For example, names like "neural networks" have always been considered metaphorical at best.

reply
terminalshort
48 minutes ago
[-]
What has it got to do with anything whether brains function like computers? This is only relevant if you define thinking as something only the brain can do, and then nothing that doesn't work like a brain can think. This would be like defining flight as "what birds do" and then saying airplanes can't fly because they don't work like birds.

And then what do you even mean by "a computer?" This falls into the same trap because it sounds like your statement that brains don't function like a computer is really saying "brains don't function like the computers I am familiar with." But this would be like saying quantum computers aren't computers because they don't work like classical computers.

reply
d-lisp
5 hours ago
[-]
Do you think that thinking is undefinable ? If thinking is definable, then all statements about it aren't unverifiable.
reply
ablob
5 hours ago
[-]
Caveat: if thinking is definable, then not all statements about it are unverifiable.
reply
d-lisp
4 hours ago
[-]
Yes, that's a problem of me not being a native english speaker. "All x aren't y" may mean "not all x are y" in my tongue. Not a single x is y is more what we would say in the previous case. But in our case we would say there are x that aren't y.

If thinking is definable, it is wrong that all statements about it are unverifiable (i.e. there are statements about it that are verifiable.)

Well, basic shit.

reply
nh23423fefe
5 hours ago
[-]
Is this some self refuting sentence?
reply
d-lisp
5 hours ago
[-]
I think they meant "Cannot evaluate : (is <undefined> like x ?), argument missing"

edit : Thinking is undefined, statements about undefined cannot be verified.

reply
ux266478
5 hours ago
[-]
is a meta-level grammar the same as an object-level grammar?
reply
random9749832
5 hours ago
[-]
Is reasoning undefined? That's what usually meant by "thinking".
reply
nutjob2
5 hours ago
[-]
Formal reasoning is defined, informal reasoning very much isn't.
reply
random9749832
5 hours ago
[-]
At the end of the day most people would agree that if something is able to solve a problem without a lookup table / memorisation that it used reasoning to reach the answer. You are really just splitting hairs here.
reply
gowld
3 hours ago
[-]
What do "most" people thinking about LLMs, then?

The "hair-splitting" underlies the whole GenAI debate.

reply
CamperBob2
2 hours ago
[-]
The difference between thinking and reasoning is that I can "think" that Elvis is still alive, Jewish space lasers are responsible for California wildfires, and Trump was re-elected president in 2020, but I cannot "reason" myself into those positions.

It ties into another aspect of these perennial threads, where it is somehow OK for humans to engage in deluded or hallucinatory thought, but when an AI model does it, it proves they don't "think."

reply
tracerbulletx
3 hours ago
[-]
Yeah. This whole AI situation has really exposed how bad most people are at considering the ontological and semantic content of the words they use.
reply
omnicognate
2 hours ago
[-]
Indeed, people confidently assert as established fact things like "brains are bound by the laws of physics" and therefore "there can't be anything special" about them, so "consciousness is an illusion" and "the mind is a computer", all with absolute conviction but with very little understanding of what physics and maths really do and do not say about the universe. It's a quasi-religious faith in a thing not fully comprehended. I hope in the long run some humility in the face of reality will eventually be (re)learned.
reply
jeffmcmahan
26 minutes ago
[-]
This. People do not understand the implications of the most basic facts of modern science. Gravitation is instantaneous action at a distance via an "occult" force (to quote Newton's contemporaries).
reply
whoknowsidont
2 hours ago
[-]
Lot's of assumptions about humanity and how unique we are constantly get paraded in this conversation. Ironically, the people who tout those perspectives are the least likely to understand why we're really not all that "special" from a very factual and academic perspective.

You'd think it would unlock certain concepts for this class of people, but ironically, they seem unable to digest the information and update their context.

reply
lisbbb
2 hours ago
[-]
A large number of adults I encounter are functionally illiterate, including business people in very high up positions. They are also almost 100% MATHEMATICALLY illiterate, not only unable to solve basic algebra and geometry problems, but completely unable to reason about statistical and probabilistic situations encountered in every day life. This is why gambling is so popular and why people are constantly fooled by politicians. It's bad enough to be without words in the modern world, but being without numbers makes you vulnerable to all manner of manipulations.
reply
sokka_h2otribe
26 minutes ago
[-]
Gambling exists more because of people dopamine systems than math...though I get the overall drift. People are fooled by politicians because ?? Also not really math related I think.
reply
snickerbockers
5 hours ago
[-]
>Equivalent statements could be made about how human brains are not magic, just biology - yet I think we still think.

They're not equivalent at all because the AI is by no means biological. "It's just maths" could maybe be applied to humans but this is backed entirely by supposition and would ultimately just be an assumption of its own conclusion - that human brains work on the same underlying principles as AI because it is assumed that they're based on the same underlying principles as AI.

reply
observationist
4 hours ago
[-]
Unless you're supposing something mystical or supernatural about how brains work, then yes, it is "just" math, there is nothing else it could be. All of the evidence we have shows it's an electrochemical network of neurons processing information. There's no evidence that suggests anything different, or even the need for anything different. There's no missing piece or deep mystery to it.

It's on those who want alternative explanations to demonstrate even the slightest need for them exists - there is no scientific evidence that exists which suggests the operation of brains as computers, as information processors, as substrate independent equivalents to Turing machines, are insufficient to any of the cognitive phenomena known across the entire domain of human knowledge.

We are brains in bone vats, connected to a wonderful and sophisticated sensorimotor platform, and our brains create the reality we experience by processing sensor data and constructing a simulation which we perceive as subjective experience.

The explanation we have is sufficient to the phenomenon. There's no need or benefit for searching for unnecessarily complicated alternative interpretations.

If you aren't satisfied with the explanation, it doesn't really matter - to quote one of Neil DeGrasse Tyson's best turns of phrase: "the universe is under no obligation to make sense to you"

If you can find evidence, any evidence whatsoever, and that evidence withstands scientific scrutiny, and it demands more than the explanation we currently have, then by all means, chase it down and find out more about how cognition works and expand our understanding of the universe. It simply doesn't look like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.

Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.

That doesn't preclude souls and supernatural appearing phenomena or all manner of "other" things happening. There's simply no need to tie it in with cognition - neurotransmitters, biological networks, electrical activity, that's all you need.

reply
johnsmith1840
2 hours ago
[-]
AI operates alot like trees do as they are both using maths under the hood.

This is the point, we don't know the delta between brains and AI any assumption is equivalent to my statement.

reply
jvanderbot
4 hours ago
[-]
Math is a superset of both processes (can model/implement both), but that doesn't imply that they are equivalent.
reply
hnfong
5 hours ago
[-]
Well, a better retort would be "Human brains are not magic, just physics. Protons, neutrons and electrons don't think".

But I think most people get what GP means.

reply
criddell
2 hours ago
[-]
Until you can define what thinking is, you can't assert that particles don't think (panpsychism).
reply
_alternator_
2 hours ago
[-]
Panpsychism is actually quite reasonable in part because it changes the questions you ask. Instead of “does it think” you need to ask “in what ways can it think, and in what ways is it constrained? What types of ‘experience/qualia’ can this system have, and what can’t it have?”

When you think in these terms, it becomes clear that LLMs can’t have certain types of experiences (eg see in color) but could have others.

A “weak” panpsychism approach would just stop at ruling out experience or qualia based on physical limitations. Yet I prefer the “strong” pansychist theory that whatever is not forbidden is required, which begins to get really interesting (would imply that for example an LLM actually experiences the interaction you have with it, in some way).

reply
pegasus
5 hours ago
[-]
But parent didn't try to apply "it's just maths" to humans. He said one could just as easily say, as some do: "Humans are just biology, hence they're not magic". Our understanding of mathematics, including the maths of transformer models is limited, just as our understanding of biology. Some behaviors of these models have taken researches by surprise, and future surprises are not at all excluded. We don't know exactly how far they will evolve.

As for applying the word thinking to AI systems, it's already in common usage and this won't change. We don't have any other candidate words, and this one is the closest existing word for referencing a computational process which, one must admit, is in many ways (but definitely not in all ways) analogous to human thought.

reply
ikrenji
5 hours ago
[-]
Human brains might not be explained by the same type of math AI is explained with, but it will be some kind of math...
reply
Mehvix
3 hours ago
[-]
There's no reason to believe this to be the case. Godel says otherwise.
reply
_alternator_
2 hours ago
[-]
Human brains and experiences seem to be constrained by the laws of quantum physics, which can be simulated to arbitrary fidelity on a computer. Nit sure where Godel’s incompleteness theory would even come in here…
reply
squidbeak
2 hours ago
[-]
Brain damage? If thought was outside physics, it would be a bit more durable than Humpty Dumpty.
reply
gowld
3 hours ago
[-]
Please explain, because this interpetation of "Godel" is highly nonstandard.
reply
Mehvix
2 hours ago
[-]
you may consider reading I am a strange loop for that, which can do far better justification than myself

if there's surely no algo to solve the halting problem, why would there be maths that describes consciousness?

reply
josh-sematic
2 hours ago
[-]
Can you look at any arbitrary program and tell if it halts without running it indefinitely? If so, you should explain how and collect your Nobel. Telling everybody whether the Collatz conjecture is correct is a good warm up. If not, you can’t solve the halting program either. What does that have to do with consciousness though?

Having read “I Am a Strange Loop” I do not believe Hofstadter indicates that the existence of Gödel’s theorem precludes consciousness being realizable on a Turing machine. Rather if I recall correctly he points out that as a possible argument and then attempts to refute it.

On the other hand Penrose is a prominent believer that human’s ability to understand Gödel’s theorem indicates consciousness can’t be realized on a Turing machine but there’s far from universal agreement on that point.

reply
AlecSchueler
5 hours ago
[-]
> that human brains work on the same underlying principles as AI

That wasn't the assumption though, it was only that human brains work by some "non-magical" electro-chemical process which could be described as a mechanism, whether that mechanism followed the same principles of AI or not.

reply
mcswell
5 hours ago
[-]
Straw man. The person who you're responding to talked about "equivalent statements" (emphasis added), whereas you appear to be talking about equivalent objects (AIs vs. brains), and pointing out the obvious flaw in this argument, that AIs aren't biology. The obvious flaw in the wrong argument, that is.
reply
cwmoore
2 hours ago
[-]
The human mind is not just biology in the same way that LLMs are just math.
reply
sounds
2 hours ago
[-]
A college level approach could look at the line between Math/Science/Physics and Philosophy. One thing from the article that stood out to me was that the introduction to their approach started with a problem about classifying a traffic light. Is it red or green?

But the accompanying XY plot showed samples that overlapped or at least were ambiguous. I immediately lost a lot of my interest in their approach, because traffic lights by design are very clearly red, or green. There aren't mauve or taupe lights that the local populace laughs at and says, "yes, that's mostly red."

I like the idea of studying math by using ML examples. I'm guessing this is a first step and future education will have better examples to learn from.

reply
krisoft
1 hour ago
[-]
> traffic lights by design are very clearly red, or green

I suspect you feel this because you are observing the output of a very sophisticated image processing pipeline in your own head. When you are dealing with raw matrixes of rgb values it all becomes a lot more fuzzy. Especially when you encounter different illuminations, exposures and the cropping of the traffic light has noise on it. Not saying it is some intractably hard machine vision problem, because it is not. But there is some variety and fuzzyness there in the raw sensor measurements.

reply
terminalshort
5 hours ago
[-]
I have yet to hear any plausible definition of "thought" that convincingly places LLMs and brains on opposite sides of it without being obviously contrived for that purpose.
reply
bruntofsaurus
2 hours ago
[-]
Define "think".

We observe through our senses geometric relationships.

Syntax is exactly that; letters, sentences, paragraphs organized in spatial/geometric relationships.

At best thought is recreation of neural networks in the brain which only exist as spatial relationships.

Our senses operate on spatial relationships; enough light to work by, and food relative to stomach to satisfy our biological impetus to survive (which is spatial relationships of biochemistry).

The idea of "thought" as anything but biology makes little sense to me then as a root source is clearly observable. Humanity, roughly, repeats the same social story. All that thought does not seem to be all that useful as we end up in the same place; the majority as serfs of aristocracy.

Personally would prefer less "thought" role-play and more people taking the load of the labor they exploit to enable them to sit and "think".

reply
smallerize
1 hour ago
[-]
There's a huge amount of money going to convincing people that AI is magic or better than people. The reprogramming is necessary.
reply
__loam
3 hours ago
[-]
We don't know how brains work.
reply
mbg721
3 hours ago
[-]
We really don't know how consciousness works. The popular theories that it's emergent might be proven correct, or might be proven to be like the idea that phlogiston built up in a vacuum, putting out flames.
reply
nutjob2
5 hours ago
[-]
AI systems compute and humans think. One is math and the other biology.

But they are two different things with overlapping qualities.

It's like MDMA and falling in love. They have many overlapping quantities but no one would claim one is the other.

reply
CamperBob2
5 hours ago
[-]
That's where these threads always end up. Someone asserts, almost violently, that AI does not and/or cannot "think." When asked how to falsify their assertion, perhaps by explaining what exactly is unique about the human brain that cannot and/or will not be possible to emulate, that's the last anyone ever hears from them. At least until the next "AI can't think" story gets posted.

The same arguments that appeared in 2015 inevitably get trotted out, almost verbatim, ten years later. It would be amusing on other sites, but it's just pathetic here.

reply
pegasus
4 hours ago
[-]
Consider that you might have become polarized yourself. I often encounter good arguments against current AI systems emulating all essential aspects of human thinking. For example, the fact that they can't learn from few examples, that they can't perform simple mathematical operations without access to external help (via tool calling) or that they have to expend so much more energy to do their magic (and yes, to me they are a bit magical), which makes some wonder if what these models do is a form of refined brute-force search, rather than ideating.

Personally, I'm ok with reusing the word "thinking", but there are dogmatic stances on both sides. For example, lots of people decreeing that biology in the end can't but reduce to maths, since "what else could it be". The truth is we don't actually know if it is possible, for any conceivable computational system, to emulate all essential aspects of human thought. There are good arguments for this (in)possibility, like those presented by Roger Penrose in "the Emperor's new Mind" and "Shadows of the Mind".

reply
CamperBob2
1 hour ago
[-]
For example, the fact that they can't learn from few examples

For one thing, yes, they can, obviously [1] -- when's the last time you checked? -- and for another, there are plenty of humans who seemingly cannot.

The only real difference is that with an LLM, when the context is lost, so is the learning. That will obviously need to be addressed at some point.

that they can't perform simple mathematical operations without access to external help (via tool calling)

But yet you are fine with humans requiring a calculator to perform similar tasks? Many humans are worse at basic arithmetic than an unaided transformer network. And, tellingly, we make the same kinds of errors.

or that they have to expend so much more energy to do their magic (and yes, to me they are a bit magical), which makes some wonder if what these models do is a form of refined brute-force search, rather than ideating.

Well, of course, all they are doing is searching and curve-fitting. To me, the magical thing is that they have shown us, more or less undeniably (Penrose notwithstanding), that that is all we do. Questions that have been asked for thousands of years have now been answered: there's nothing special about the human brain, except for the ability to form, consolidate, consult, and revise long-term memories.

1: E.g., https://arxiv.org/abs/2005.14165 from 2020

reply
ablob
4 hours ago
[-]
Usually it is the work of the one claiming something to prove it. So if you believe that AI does "think" you are expected to show me that it really does. Claiming it "thinks - prove otherwise" is just bad form and also opens the discussion up for moving the goalposts just as you did with your brain emulation statement. Or you could just not accept any argument made or circumvent it by stating the one trying to disprove your assertion got the definition wrong. There are countless ways to start a bad faith argument using this methodology, hence: Define property -> prove property.

Conversely, if the one asserting something doesn't want to define it there is no useful conversation to be had. (as in: AI doesn't think - I won't tell you what I mean by think)

PS: Asking someone to falsify their own assertion doesn't seem a good strategy here.

PPS: Even if everything about the human brain can be emulated, that does not constitute progress for your argument, since now you'd have to assert that AI emulates the human brain perfectly before it is complete. There is no direct connection between "This AI does not think" to "The human brain can be fully emulated". Also the difference between "does not" and "can not" is big enough here that mangling them together is inappropriate.

reply
CamperBob2
1 hour ago
[-]
So if you believe that AI does "think" you are expected to show me that it really does.

A lot of people seemingly haven't updated their priors after some of the more interesting results published lately, such as the performance of Google's and OpenAI's models at the 2025 Math Olympiad. Would you say that includes yourself?

If so, what do the models still have to do in order to establish that they are capable of all major forms of reasoning, and under what conditions will you accept such proof?

reply
Tadpole9181
3 hours ago
[-]
Then prove to me you are thinking, lest we assume you are a philosophical zombie and need no rights or protections.

Sometimes, because of the consequences of otherwise, the order gets reversed

reply
ablob
1 hour ago
[-]
Well first of all I never claimed that I was capable of thinking (smirk). We also haven't agreed on a definition of "thinking" yet, so as you can read in my previous comment, there is no meaningful conversation to be had. I also don't understand how your oddly aggresive phrasing adds to the conversation, but if it helps you: my rights and protections do not depend on whether I'm able to prove to you that I am thinking. (It also derails the conversation for what it's worth - it's a good strategy in the debating club, but these are about winning or loosing and not about fostering and obtaining knowledge)

Whatever you meant to say with "Sometimes, because of the consequences of otherwise, the order gets reversed" eludes me as well.

reply
Tadpole9181
45 minutes ago
[-]
If I say I'm innocent, you don't say I have to prove it. Some facts are presumed to be true without burden of evidence because otherwise it could cause great harm.

So we don't require, say, minorities or animals to prove they have souls, we just inherently assume they do and make laws around protecting them.

reply
Terr_
5 hours ago
[-]
Someone asserts, almost religiously, that LLMs do and/or can "think." When asked how to falsify their assertion, perhaps by explaining what exactly is "thinking" in the human brain that can and/or will be possible to emulate...
reply
gfdvgfffv
5 hours ago
[-]
One mostly sees people aggressively claiming they can’t, ever. On the other side people seem to simply allow that they might, or might eventually.
reply
umanwizard
5 hours ago
[-]
Err, no, that’s not what’s happening. Nobody, at least in this thread (and most others like it I’ve seen), is confidently claiming LLMs can think.

There are people confidently claiming they can’t and then other people expressing skepticism at their confidence and/or trying to get them to nail down what they mean.

reply
jayveeone
2 hours ago
[-]
This entire debate over the years is because so many confidently assert that AI can think, or that AI will soon be our God, or our ruler, etc.
reply
superkuh
5 hours ago
[-]
Or they just point to the turing test which was the defacto standard test for something so nebulous. And behold: LLM can pass the turing test. So they think. Can you come up with something better (than the turing test)?
reply
bigfishrunning
2 hours ago
[-]
But the Turing test (which I concede, LLMs do pass) doesn't test if some system is thinking; it tests if the system can convince an unbiased observer that it is thinking. I cannot come up with a better "is this thing thinking" test, but that doesn't mean that such a test can't exist; I'm sure there are much smarter people then me trying to solve this problem.
reply
CamperBob2
5 hours ago
[-]
When asked how to falsify their assertion, perhaps by explaining what exactly is "thinking" in the human brain that can and/or will be possible to emulate...

... someone else points out that the same models that can't "think" are somehow turning in gold-level performance at international math and programming competitions, making Fields Medalists sit up and take notice, winning art competitions, composing music indistinguishable from human output, and making entire subreddits fail the Turing test.

reply
Terr_
5 hours ago
[-]
A couple decades of chess programs nods knowingly: "First time?"
reply
CamperBob2
5 hours ago
[-]
A couple decades of chess programs nods knowingly: "First time?"

Uh huh. Good luck getting Stockfish to do your math homework while Leela works on your next waifu.

LLMs play chess poorly. Chess engines do nothing else at all. That's kind of a big difference, wouldn't you say?

reply
ben_w
4 hours ago
[-]
> That's kind of a big difference, wouldn't you say?

To their utility.

Not sure if it matters on the question "thinking?"; even if for the debaters "thinking" requires consciousness/qualia (and that varies), there's nothing more than guesses as to where that emerges from.

reply
gowld
3 hours ago
[-]
Terr_ was agreeing with you and highlighting how old the debate is.
reply
Terr_
2 hours ago
[-]
Highlighting, yes, agreeing, no.

For my original earlier reply, the main subtext would be: "Your complaint is ridiculously biased."

For the later reply about chess, perhaps: "You're asserting that tricking, amazing, or beating a human is a reliable sign of human-like intelligence. We already know that is untrue from decades of past experience."

reply
CamperBob2
1 hour ago
[-]
You're asserting that tricking, amazing, or beating a human is a reliable sign of human-like intelligence.

I don't know who's asserting that (other than Alan Turing, I guess); certainly not me. Humans are, if anything, easier to fool than our current crude AI models are. Heck, ELIZA was enough to fool non-specialist humans.

In any case, nobody was "tricked" at the IMO. What happened there required legitimate reasoning abilities. The burden of proof falls decisively on those who assert otherwise.

reply
nh23423fefe
5 hours ago
[-]
god of the gaps
reply
CamperBob2
5 hours ago
[-]
Exactly. As soon as a model does something it "wasn't supposed to be able to do," two gaps open up on either side.
reply
nutjob2
5 hours ago
[-]
Computers can perform math and numerous other tasks billions of times faster than humans, whats your point?

This is exactly the problem. Claims about AI are unfalsifiable, thus your various non-sequiturs about AI 'thinking'.

reply
IgorPartola
2 hours ago
[-]
I feel like these conversations really miss the mark: whether an LLM thinks or not is not a relevant question. It is a bit like asking “what color is an Xray?” or “what does the number 7 taste like?”

The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being. It is a building block. Your brain thinks. Your prefrontal cortex however is not a complete system and if you somehow managed to extract it and wire it up to a serial terminal I suspect you’d be pretty disappointed in what it would be capable of on its own.

I want to be clear that I am not making an argument that once we hook up sensory inputs and motion outputs as well as motivations, fears, anxieties, desires, pain and pleasure centers, memory systems, sense of time, balance, fatigue, etc. to an LLM that we would get a thinking feeling conscious being. I suspect it would take something more sophisticated than an LLM. But my point is that even if an LLM was that building block, I don’t think the question of whether it is capable of thought is the right question.

reply
nkrisc
1 hour ago
[-]
> The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being.

The AI companies themselves are the ones drawing the parallels to a human being. Look at how any of these LLM products are marketed and described.

reply
djeastm
1 hour ago
[-]
Is it not within our capacity on HN to ignore whatever the marketers say and speak to the underlying technology?
reply
SiempreViernes
1 hour ago
[-]
There was around 500 comments on the OpenAI + Disney story, so the evidence points to "no".
reply
ChuckMcM
1 hour ago
[-]
Why do people call is "Artificial Intelligence" when it could be called "Statistical Model for Choosing Data"?

"Intelligence" implies "thinking" for most people, just as "Learning" in machine learning implies "understanding" for most people. The algorithms created neither 'think' nor 'understand' and until you understand that, it may be difficult to accurately judge the value of the results produced by these systems.

reply
james_marks
1 hour ago
[-]
If we say “artificial flavoring”, we have a sense that it is an emulation of something real, and often a poor one.

Why, when we use the term for AI, do we skip over this distinction and expect it to be as good as the original—- or better?

That wouldn’t be artificial intelligence, it would just be the original artifact: “intelligence”.

reply
wongarsu
48 minutes ago
[-]
The term was coined in 1955 to describe "learning or any other feature of intelligence" simulated by a machine [1]. The same proposal does list using natural language as one of the aspects of "the artificial intelligence problem"

It's not a perfect term, but we have been using it for seven full decades to include all of machine learning and plenty of things even less intelligent

1: https://www-formal.stanford.edu/jmc/history/dartmouth/dartmo...

reply
resonious
57 minutes ago
[-]
Actually I think the name is apt. It's artificial. It's like how an "artificial banana" isn't actually a banana. It doesn't have to be real thinking or real learning, it just has to look intelligent (which it does).
reply
IgorPartola
1 hour ago
[-]
How do you feel about Business Intelligence as a term?
reply
ChuckMcM
1 hour ago
[-]
Same way I feel about 'Military Intelligence' :-). Both of those phrases use the 'information gathering and analysis' definition of intelligence rather than the 'thinking' definition.
reply
mjcohen
48 minutes ago
[-]
Like the old saying:

Military justice is to justice as military music is to music

reply
SiempreViernes
1 hour ago
[-]
You should take your complaints to OpenAI, who constantly write like LLMs think in the exact same sense as humans; here a random example:

> Large language models (LLMs) can be dishonest when reporting on their actions and beliefs -- for example, they may overstate their confidence in factual claims or cover up evidence of covert actions

reply
tehjoker
1 hour ago
[-]
They have a product to sell based on the idea AGI is right around the corner. You can’t trust Sam Altman as far as you can throw him.

Still, the sales pitch has worked to unlock huge liquidity for him so there’s that.

Still making predictions is a big part of what brains do though not the only thing. Someone wise said that LLM intelligence is a new kind of intelligence, like how animal intelligence is different from ours but is still intelligence but needs to be characterized to understand differences.

reply
SiempreViernes
1 hour ago
[-]
> Someone wise said that LLM intelligence is a new kind of intelligence

So long as you accept the slide ruler as a "new kind of intelligence" everything will probably work out fine, it's the Altmannian insistence that only the LLM is of the new kind that is silly.

reply
wisty
16 minutes ago
[-]
This is the underlying problem behind syncophantcy.

I saw a YouTube video about a investigative youtuber Eddy Burback who very easily convinced chat4 that he should cut off all contact with friends and family, move to a cabin in the desert, eat baby food, wrap himself in alfoil, etc just feeding his own (faked) mistakes and delusions. "What you are doing is important, trust your instincts".

Wven if AI could hypothetically be 100x as smart as a human under the hood, it still doesn't care. It doesn't do what it thinks it should, it doesn't do what it needs to do, it does what we train it to.

We train in humanities weaknesses and follies. AI can hypothetically exceed humanity in some respects, but in other respects it is a very hard to control power tool.

AI is optimised, and optimised functions always "hack" the evaluation function. In the case of AI, the evaluation function includes human flaws. AI is trained to tell us what we want to hear.

Elon Musk sees the problem, but his solution is to try to make it think more like him, and even if that succeeds it just magnifies his own weaknesses.

Has anyone read the book criticising Ray Dalio? He is a very successful hedge fund manager, who decided that he could solve the problem of finding a replacement by psychology evaluation and training people to think like him. But even his smartest employees didn't think like him, they just (reading between the lines) gamed his system. Their incentives weren't his incentives - he could demand radical honesty and integrity but that doesn't work so well when he would (of course) reward the people who agreed with him, rather than the people who would tell him he was screwing up. His organisation (apparently) became a bunch of even more radical syncopants due to his efforts to weed out syncophantcy.

reply
sublinear
1 hour ago
[-]
The part that's most infuriating is that we don't have to speculate at all. Any discussion or philosophizing beyond the literal computer science is simply misinformation.

There's absolutely no similarity between what computer hardware does and what a brain does. People will twist and stretch things and tickle the imagination of the naive layperson and that's just wrong. We seriously have to cut this out already.

Anthropomorphizing is dangerous even for other topics, and long understood to be before computers came around. Why do we allow this?

The way we talk about computer science today sounds about as ridiculous as invoking magic or deities to explain what we now consider high school physics or chemistry. I am aware that the future usually sees the past as primitive, but why can't we try to seem less dumb at least this time around?

reply
Kim_Bruning
31 minutes ago
[-]
> There's absolutely no similarity between what computer hardware does and what a brain does.

But at very least there's also no similarity between what computer hardware does and what even the simplest of LLMs do. They don't run on eg. x86_64 , else qemu would be sufficient for inferencing.

reply
pitaj
40 minutes ago
[-]
Similarity of the hardware is absolutely irrelevant when we're talking about emergent behavior like "thought".
reply
terminalshort
34 minutes ago
[-]
Given that the headline is:

> Secondary school maths showing that AI systems don’t think

And the article contains the quotes:

> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths.

> The team is taking very complex ideas and reducing them to such an extent that we can use secondary classroom maths to show that AI is not magic and AI systems do not think.

This is not off topic

reply
t23414321
1 hour ago
[-]
Wouldn't 'thinking' need to be updating the model of reality (LLM is not yet that, just words) - at every step doing again all that extensive calculations as when/to creating/approximating that/better model (learning) ?

Expecting machines to think is.. like magical thinking (but they are good at calculations indeed).

I wish we didn't use the word intelligence in context of LLMs - shortly there is Essence and the rest.. is only slope - into all possible combinations of Markov Chains - may they have sense or not I don't see how part of some calculation could recognize it, or that to be possible from inside (of calculation, that doesn't even consider that).

Aside of artificial knowledge (out of senses, experience, context lengths.. - confabulating but not knowing that), I wish to see an intelligent knowledge - made in kind of semantic way - allowed to expand using not yet obvious (but existing - not random) connections. I wouldn't expect it to think (humans think, digitals calculate). But I would expect it to have a tendency to be coming closer (not further) in reflecting/modeling reality and expanding implications.

reply
Retric
1 hour ago
[-]
Thinking is different than forming long term memories.

An LLM could be thinking in one of two ways. Either between adding each individual token, or collectively across multiple tokens. At the individual token level the physical mechanism doesn’t seem to fit the definition being essentially reflexive action, but across multiple tokens that’s a little more questionable especially as multiple approaches are used.

reply
t23414321
56 minutes ago
[-]
An LLM ..is calculated ..from language (or from things being said by humans before being true or not). It's not some antropomorfic process what using the word thinking would suggest (to sell well).

> across multiple tokens

- but how many ? how many of them happen in sole person life ? How many in some calculation ? Does it matter, if a calculation doesn't reflect it but stay all the same ? (conversation with.. a radio - would it have any sense ?)

reply
frozenlettuce
5 hours ago
[-]
You can replicate all calculations done by LLMs with pen and paper. It would take ages to calculate anything, but it's possible. I don't think that pen and paper will ever "think", regardless of how complex the calculations involved are.
reply
gus_massa
5 hours ago
[-]
The official name is https://en.wikipedia.org/wiki/Chinese_room

The opinions are exactly the same than about LLM.

reply
circuit10
1 hour ago
[-]
I don’t understand why people say the Chinese Room thing would prove LLMs don’t think, to me it’s obvious that the person doesn’t understand Chinese but the process does, similarly the CPU itself doesn’t understand the concepts an LLM can work with but the LLM itself does, or a neuron doesn’t understand concepts but the entire structure of your brain does

The concept of understanding emerges on a higher level from the way the neurons (biological or virtual) are connected, or the way the instructions being followed by the human in the Chinese room process the information

But really this is a philosophical/definitional thing about what you call “thinking”

Edit: I see my take on this is listed on the page as the “System reply”

reply
sigmoid10
5 hours ago
[-]
And the counter argument is also exactly the same. Imagine you take one neuron from a brain and replace it with an artificial piece of electronics (e.g. some transistors) that only generates specific outputs based on inputs, exactly like the neuron does. Now replace another neuron. And another. Eventually, you will have the entire brain replaced with a huge set of fundamentally super simple transistors. I.e. a computer. If you believe that consciousness or the ability to think disappears somewhere during this process, then you are essentially believing in some religious meta-physics or soul-like component in our brains that can not be measured. But if it can not be measured, it fundamentally can not affect you in any way. So it doesn't matter for the experiment in the end, because the outcome would be exactly the same. The only reason you might think that you are conscious and the computer is not is because you believe so. But to an outsider observer, belief is all it is. Basically religion.
reply
kipchak
4 hours ago
[-]
It seems like the brain "just" being a giant number of neurons is an assumption. As I understand it's still an area of active research, for example the role of glial cells. The complete function may or may not be pen and paper-able.
reply
Tadpole9181
3 hours ago
[-]
> The complete function may or may not be pen and paper-able.

Would you mind expanding on this? At a base read, it seems you implying magic exists.

reply
kipchak
23 minutes ago
[-]
It could well be the case that the brain can be simulated, but presently we don't know exactly what variables/components must be simulated. Does ongoing neuroplasticity for example need to be a component of simulation? Is there some as of yet unknown causal mechanisms or interactions that may be essential?
reply
bigfishrunning
2 hours ago
[-]
> component in our brains that can not be measured.

"Can not be measured", probably not. "We don't know how to measure", almost certainly.

I am capable of belief, and I've seen no evidence that the computer is. It's also possible that I'm the only person that is conscious. It's even possible that you are!

reply
danaris
4 hours ago
[-]
But you are now arguing against a strawman, namely, "it is not possible to construct a computer that thinks".

The argument that was actually made was "LLMs do not think".

reply
umanwizard
4 hours ago
[-]
A: X, because Y

B: But Y would also imply Z

C: A was never arguing for Z! This is a strawman!

reply
danaris
4 hours ago
[-]
"LLMs cannot think like brains" does not imply "no computer it will ever be possible to construct could think like a brain".
reply
umanwizard
4 hours ago
[-]
“LLMs cannot think like brains” is “X”.
reply
mcswell
5 hours ago
[-]
I don't see the relevance of that argument (which other responders to your post have pointed out as Searle's Chinese Room argument). The pen and paper are of course not doing any thinking, but then the pen isn't doing any writing on its own, either. It's the system of pen + paper + human that's doing the thinking.
reply
frozenlettuce
2 hours ago
[-]
The idea of my argument is that I notice that people project some "ethereal" properties over computations that happen in the... computer. Probably because electricity is involved, making things show up as "magic" from our point of view, making it easier to project consciousness or thinking onto the device. The cloud makes that even more abstract. But if you are aware that the transistors are just a medium that replicates what we already did for ages with knots, fingers, and paint, it gets easier to see them as plain objects. Even the resulting artifacts that the machine produces are only something meaningful from our point of view, because you need prior knowledge to read the output signals. So yeah, those devices end up being an extension of ourselves.
reply
hackinthebochs
48 minutes ago
[-]
Your view is missing the forest for the trees. You see individual objects but miss the aggregate whole. You have a hard time conceiving of how exotic computers can be conscious because we are scale chauvinists by design. Our minds engage with the world on certain time and length scales, and so we naturally conceptualize our world based on entities that exist on those scales. But computing is necessarily scale independent. It doesn't matter to the computation if it is running on some 100GHz substrate or .0001Hz. It doesn't matter if its running on a CPU chip the size of a quarter or spread out over the entire planet. Computation is about how information is transformed in semantically meaningful ways. Scale just doesn't matter.

If you were a mind supervening on the behavior of some massive time/space scale computer, how would you know? How could you tell the difference between running on a human making marks with pen and paper and running on a modern CPU? Your experience updates based on information transformations, not based on how fast the fundamental substrate is changing. When your conscious experience changes, that means your current state is substantially different from your prior state and you can recognize this difference. Our human-scale chauvinism gets in the way of properly imagining this. A mind running on a CPU or a large collection of human computers is equally plausible.

A common question people like to ask is "where is the consciousness" in such a system. This is an important question if only because it highlights the futility of such questions. Where is Microsoft Word when it is running on my computer? How can you draw a boundary around a computation when there are a multitude of essential and non-essential parts of the system that work together to construct the relevant causal dynamic. It's just not a well-defined question. There is no one place where Microsoft Word occurs nor is there any one place where consciousness occurs in a system. Is state being properly recorded and correctly leveraged to compute the next state? The consciousness is in this process.

reply
BobbyJo
2 hours ago
[-]
If you put a droplet of water in a warm bowl every 12 hours, the bowl will remain empty as the water will evaporate. That does not mean that if you put a trillion droplets in every twelve hours it will still remain empty.
reply
SiempreViernes
1 hour ago
[-]
It will also not be empty if I put the bowl in the sea, which is a remark about the nature of thoughthat that proves exactly as much as your comment.
reply
Wowfunhappy
5 hours ago
[-]
https://xkcd.com/505/

You can replicate the entire universe with pen and paper (or a bunch of rocks). It would take an unimaginably long time, and we haven't discovered all the calculations you'd need to do yet, but presumably they exist and this could be done.

Does that actually make a universe? I don't know!

The comic is meant to be a joke, I think, but I find myself thinking about it all the time!!!

reply
frozenlettuce
1 hour ago
[-]
Even worse, as we are part of the universe, we would need to simulate ourselves and the very simulation that we are creating. You would also need to replicate the simulation of the simulation, leading to an eternal loop that would demand infinite matter and time (and would still not be enough!). Probably, you can't simulate something while being part of it.
reply
Wowfunhappy
1 hour ago
[-]
It doesn’t need to be our universe, just a universe.

The question is, are the people in the simulated universe real people? Do they think and feel like we do—are they conscious? Either answer seems like it can’t possibly be right!

reply
umanwizard
5 hours ago
[-]
You can simulate a human brain on pen and paper too.
reply
palmotea
5 hours ago
[-]
> You can simulate a human brain on pen and paper too.

That's an assumption, though. A plausible assumption, but still an assumption.

We know you can execute an LLM on pen and paper, because people built them and they're understood well enough that we could list the calculations you'd need to do. We don't know enough about the human brain to create a similar list, so I don't think you can reasonably make a stronger statement than "you could probably simulate..." without getting ahead of yourself.

reply
terminalshort
5 hours ago
[-]
I can make a claim much stronger than "you could probably" The counterclaim here is that the brain may not obey physical laws that can be described by mathematics. This is a "5G causes covid" level claim. The overwhelming burden of proof is on you.
reply
frozenlettuce
1 hour ago
[-]
There are some quantum effects in the brain (for some people, that's a possible source of consciousness). We can simulate quantum effects, but here comes the tricky part: even if our simulation matches the probability, say 70/30 of something happening, what guarantees that our simulation would take the same path as the object being simulated?
reply
daedrdev
59 minutes ago
[-]
We don't have to match the quantum state since the brain still produces an valid output regardless of what each random quantum state ended up on. After all we can include random entropy in a LLM too.
reply
terminalshort
36 minutes ago
[-]
This is just non-determinism. Not only can't your simulation reproduce the exact output, but neither can your brain reproduce its own previous state. This doesn't mean it's a fundamentally different system.
reply
kipchak
4 hours ago
[-]
Consider for example Orch OR theory. If it or something like it were to be accurate, the brain would not "obey physical laws that can be described by mathematics".
reply
bondarchuk
2 hours ago
[-]
>Consider for example Orch OR theory

Yes, or what about leprechauns?

reply
hnfong
5 hours ago
[-]
This is basically the Church-Turing thesis and one of the motivations of using tape(paper) and an arbitrary alphabet in the Turing machine model.

It's been kinda discussed to oblivion in the last century, interesting that it seems people don't realize the "existing literature" and repeat the same arguments (not saying anyone is wrong).

reply
phantasmish
5 hours ago
[-]
The simulation isn't an operating brain. It's a description of one. What it "means" is imposed by us, what it actually is, is a shitload of graphite marks on paper or relays flipping around or rocks on sand or (pick your medium).

An arbitrarily-perfect simulation of a burning candle will never, ever melt wax.

An LLM is always a description. An LLM operating on a computer is identical to a description of it operating on paper (if much faster).

reply
gnull
5 hours ago
[-]
What makes the simulation we live in special compared to the simulation of a burning candle that you or I might be running?

That simulated candle is perfectly melting wax in its own simulation. Duh, it won't melt any in ours, because our arbitrary notions of "real" wax are disconnected between the two simulatons.

reply
hnfong
5 hours ago
[-]
They do have a valid subtle point though.

If we don't think the candle in a simulated universe is a "real candle", why do we consider the intelligence in a simulated universe possibly "real intelligence"?

Being a functionalist ( https://en.wikipedia.org/wiki/Functionalism_(philosophy_of_m... ) myself, I don't know the answer on the top of my head.

reply
hackinthebochs
16 minutes ago
[-]
>If we don't think the candle in a simulated universe is a "real candle", why do we consider the intelligence in a simulated universe possibly "real intelligence"?

I can smell a "real" candle, a "real" candle can burn my hand. The term real here is just picking out a conceptual schema where its objects can feature as relata of the same laws, like a causal compatibility class defined by a shared causal scope. But this isn't unique to the question of real vs simulated. There are causal scopes all over the place. Subatomic particles are a scope. I, as a particular collection of atoms, am not causally compatible with individual electrons and neutrons. Different conceptual levels have their own causal scopes and their own laws (derivative of more fundamental laws) that determine how these aggregates behave. Real (as distinct from simulated) just identifies causal scopes that are derivative of our privileged scope.

Consciousness is not like the candle because everyone's consciousness is its own unique causal scope. There are psychological laws that determine how we process and respond to information. But each of our minds are causally isolated from one another. We can only know of each other's consciousness by judging behavior. There's nothing privileged about a biological substrate when it comes to determining "real" consciousness.

reply
phantasmish
4 hours ago
[-]
I think the core trouble is that it's rather difficult to simulate anything at all without requiring a human in the loop before it "works". The simulation isn't anything (well, it's something, but it's definitely not what it's simulating) until we impose that meaning on it. (We could, of course, levy a similar accusation at reality, but folks tend to avoid that because it gets uselessly solipsistic in a hurry)

A simulation of a tree growing (say) is a lot more like the idea of love than it is... a real tree growing. Making the simulation more accurate changes that not a bit.

reply
penteract
1 hour ago
[-]
I believe that the important part of a brain is the computation it's carrying out. I would call this computation thinking and say it's responsible for consciousness. I think we agree that this computation would be identical if it were simulated on a computer or paper. If you pushed me on what exactly it means for a computation to physically happen and create consciousness, I would have to move to statements I'd call dubious conjectures rather than beliefs - your points in other threads about relying on interpretation have made me think more carefully about this.

Thanks for stating your views clearly. I have some questions to try and understand them better:

Would you say you're sure that you aren't in a simulation while acknowledging that a simulated version of you would say the same?

What do you think happens to someone whose neurons get replaced by small computers one by one (if you're happy to assume for the sake of argument that such a thing is possible without changing the person's behavior)?

reply
amelius
54 minutes ago
[-]
Here is a thought experiment:

Build a simulation of creatures that evolve from simple structures (think RNA, DNA).

Now, if in this simulation, after many many iterations, the creatures start talking about consciousness, what does that tell us?

reply
cibyr
2 hours ago
[-]
It seems to me that the distinction becomes irrelevant as soon as you connect inputs and outputs to the real world. You wouldn't say that a 737 autopilot can never, ever fly a real jet and yet it behaves exactly the same whether it's up in the sky or hooked up to recorded/simulated signals on a test bench.
reply
amelius
2 hours ago
[-]
> An arbitrarily-perfect simulation of a burning candle will never, ever melt wax.

It might if the simulation includes humans observing the candle.

reply
pton_xd
5 hours ago
[-]
So the brain is a mathematical artifact that operates independently from time? It just happens to be implemented using physics? Somehow I doubt it.
reply
thrance
5 hours ago
[-]
The brain follows the laws of physics. The laws of physics can be closely approximated by mathematical models. Thus, the brain can be closely approximated by mathematical models.
reply
andrepd
5 hours ago
[-]
It's an open problem whether you can or not.
reply
space_fountain
5 hours ago
[-]
It’s not that open. We can simulate smaller system of neurons just fine, we can simulate chemistry. There might be something beyond that in our brains for some reason, but it sees doubtful right now
reply
phantasmish
5 hours ago
[-]
Our brains actually do something, may be the difference. They're a thing happening, not a description of a thing happening.

Whatever that something that it actually does in the real, physical world is produces the cogito in cogito, ergo sum and I doubt you can get it just by describing what all the subatomic particles are doing, any more than a computer or pen-and-paper simulated hurricane can knock your house down, no matter how perfectly simulated.

reply
ehsanu1
5 hours ago
[-]
Doing something merely requires I/O. Brains wouldn't be doing much without that. A sufficiently accurate simulation of a fundamentally computational process is really just the same process.
reply
terminalshort
5 hours ago
[-]
Why are the electric currents moving in a GPU any less of a "thing happening" than the firing of the neurons in your brain? What you are describing here is a claim that the brain is fundamentally supernatural.
reply
phantasmish
4 hours ago
[-]
Thinking that making scribbles that we interpret(!!!) as perfectly describing a functioning consciousness and its operation, on a huge stack of paper, would manifest consciousness in any way whatsoever (hell, let's say we make it an automated flip-book, too, so it "does something"), but if you made the scribbles slightly different it wouldn't work(!?!? why, exactly, not ?!?!), is what's fundamentally supernatural. It's straight-up Bronze Age religion kinds of stuff (which fits—the tech elite is full of that kind of shit, like mummification—er, I mean—"cryogenic preservation", millenarian cults er, I mean The Singularity, et c)

Of course a GPU involves things happening. No amount of using it to describe a brain operating gets you an operating brain, though. It's not doing what a brain does. It's describing it.

(I think this is actually all somewhat tangential to whether LLMs "can think" or whatever, though—but the "well of course they might think because if we could perfectly describe an operating brain, that would also be thinking" line of argument often comes up, and I think it's about as wrong-headed as a thing can possibly be, a kind of deep "confusing the map for the territory" error; see also comments floating around this thread offhandedly claiming that the brain "is just physics"—like, what? That's the cart leading the horse! No! Dead wrong!)

reply
thrance
5 hours ago
[-]
You're arguing for the existence of a soul, for dualism. Nothing wrong with that, except we have never been able to measure it, and have never had to use it to explain any phenomenon of the brain's working. The brain follows the rules of physics, like any other objects of the material world.

A pen and paper simulation of a brain would also be "a thing happening" as you put it. You have to explain what is the magical ingredient that makes the brain's computations impossible to replicate.

You could connect your brain simulation to an actual body, and you'd be unable to tell the difference with a regular human, unless you crack it open.

reply
phantasmish
5 hours ago
[-]
> You're arguing for the existence of a soul, for dualism.

I'm not. You might want me to be, but I'm very, very much not.

reply
an0malous
5 hours ago
[-]
Parent said replicate, as in deterministically
reply
thrance
5 hours ago
[-]
You're arguing against Functionalism [0], of which I'd encourage you to at least read the Wikipedia page. Why would doing the brain's computations on pen and paper rather than on wetware lead to different outcomes? And how?

Connect your pen and paper operator to a brainless human body, and you got something indistinguishable from a regular alive human.

[0] https://en.wikipedia.org/wiki/Functionalism_%28philosophy_of...

reply
t23414321
31 minutes ago
[-]
@dang offtopicness started from using word thinking in place of calculating what is the common objection in this thread.
reply
softwaredoug
1 hour ago
[-]
LLMs don't really think, they emulate their training data. Which has a lot of examples of humans walking through problems to arrive at an answer. So naturally, if we prompt an LLM to do the same, it will emulate those examples (which tend to be more correct).

LLMs are BAD at evaluating earlier thinking errors, precisely because there's not copious examples of text where humans thinking through a problem, screwing up, going back, correcting their earlier statement, and continuing. (a good example catches these and corrects them)

reply
terminalshort
39 minutes ago
[-]
Claude code is actually great at that
reply
nevertoolate
3 hours ago
[-]
- how to prove that humans can argue endlessly like an llm?

- ragebait them by saying AIs don’t think

- …

reply
kenjackson
53 minutes ago
[-]
This article doesn’t really show anything near what the title assets.
reply
WhyOhWhyQ
5 hours ago
[-]
A lot of the drama here is due to the ambiguity of what the word 'think' is supposed to mean. One camp associates 'thinking' to consciousness, another does not. I personally believe it is possible to create an animal-like or human-like intelligence, without consciousness existing in the system. I personally would still describe whatever processing that system is doing as 'thinking'. Others believe in "substrate independence"; they think any such system must be consciousness.

(Sneaking a bit of belief in here, to me "substrate independence" is a more extreme position than the idea that a system could be made which is intelligent but not conscious, hence I find it implausible.)

reply
hamdingers
5 hours ago
[-]
If it comes to the correct answer I don't particularly care how it got there.
reply
emp17344
5 hours ago
[-]
In most cases, you don’t know if it came to the correct answer.
reply
hamdingers
2 hours ago
[-]
In every reasonable use case for LLMs verifying the answer is trivial. Does the code do what I wanted it to? Does it link to a source that corroborates the response?

If you're asking for things you can't easily verify you're barking up the wrong tree.

reply
ares623
5 hours ago
[-]
How do you know if it came to the right answer?
reply
mcswell
5 hours ago
[-]
It's not always the case, but often verifying an answer is far easier than coming up with the answer in the first place. That's precisely the principle behind the RSA algorithm for cryptography.
reply
downboots
2 hours ago
[-]
Sure, it's easy to check ((sqrt(x-3)+1)/(x/8)) is less than 4. Now do it without calculus.

Very much like this effect https://www.reddit.com/r/opticalillusions/comments/1cedtcp/s... . Shouldn't hide complexity under a truth value.

reply
terminalshort
6 hours ago
[-]
> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths

This is completely idiotic. Do these people actually believe that showing it can't be actual thought because it is described by math?

reply
nomel
5 hours ago
[-]
Wait until they hear about the physics/maths related to neurons firing!
reply
startupsfail
1 hour ago
[-]
<think>Ok, the user is claiming that... </think> ....
reply
brador
5 hours ago
[-]
Do we think?

By every scientific measure we have the answer is no. It’s just electrical current taking the path of least resistance through connected neurons mixed with cell death.

The fact a human brain peaks at IQ around 200 is fascinating. Can the scale even go higher? It would seem no since nothing has achieved a higher score it must not exist.

reply
bigfishrunning
2 hours ago
[-]
The IQ scale is constantly adjusted to keep the peak of the curve at 100 and the standard deviation around 15. To say it peaks around 200 is a pretty gross misunderstanding of what IQ means.
reply
ares623
3 hours ago
[-]
3 years ago this is the kind of posts that end up in /r/im14andthisisdeep
reply
alanuhoo
5 hours ago
[-]
[flagged]
reply
dang
39 minutes ago
[-]
Maybe so, but please don't post unsubstantive comments to Hacker News.
reply
josefritzishere
3 hours ago
[-]
I think we all intuitively knew this but it's pretty cool.
reply