> Most notably, it provides confidence levels in its findings, which Cheeseman emphasizes is crucial.
These 'confidence levels' are suspect. You can ask Claude today, "What is your confidence in __" and it will, unsurprisingly, give a 'confidence interval'. I'd like to better understand the system implemented by Cheeseman. Otherwise I find the whole thing, heh, cheesy!
When asked about their confidence, these things are almost entirely useless. If the Magic Disruption Box is incapabele of knowing whether or not it read "42/A" correctly, I'm not convinced it's gonna revolutionize science by doing autonomous research.
If you give the model the image and a prior prediction, what can it tell you? Asking for it to produce a 1-10 figure in the same token stream as the actual task seems like a flawed strategy.
There should be some research results showing their fundamental limitations. As opposed to empirical observations. Can you point at them?
What about VLMs, VLAs, LMMs?
However you feel about LLMs, and I say this because you don't have to use them for very long before you witness how useful they can be for large datasets so I'm guessing you're not a fan, they are undeniably incredible tools in some areas of science.
https://news.stanford.edu/stories/2025/02/generative-ai-tool...
Not disagreeing with your initial statement about LLMs being good and finding patterns in datasets btw.
The first developed a model to calculate protein function based on DNA sequence - yet provides no results of testing of the model. Until it does, it’s no better than the hundreds of predictive models thrown on the trash heap of science.
The second tested a models “ability to predict neuroscience results” (which reads really oddly). How did they test it? Pitted humans against LLMs in determining which published abstracts were correct.
Well yeah? That’s exactly what LLMs are good at - predicting language. But science is not advanced by predicting which abstracts of known science are correct.
It reminds me of my days in working with computational chemists - we had an x-ray structure of the molecule bound to the target. You can’t get much better than that at hard, objective data.
“Oh yeah, if you just add a methyl group here you’ll improve binding by an order of magnitude”.
So we went back to the lab, spent a week synthesizing the molecule, sent it to the biologists for a binding study. And the new molecule was 50% worse at binding.
And that’s not to blame the computation chemist. Biology is really damn hard. Scientists are constantly being surprised at results that are contradictory to current knowledge.
Could LLMs be used in the future to help come up with broad hypotheses in new areas? Sure! Are the hypotheses going to prove fruitless most of the time? Yes! But that’s science.
But any claim of a massive leap in scientific productivity (whether LLMs or something else) should be taken with a grain of salt.
Where by "good at" you mean "are totally shit at"?
They routinely hallucinate things even on tiny datasets like codebases.
But the latter doesn't invalidate the former.
Citation needed?
Closest I've seen to that was Dario saying AI would write 90% of the code, but that's very different from declaring the death of software development as an occupation.
Even if the article is accurate, it still makes sense to question the motives of the publisher. Especially if they’re selling a product.
> Cheeseman finds Claude consistently catches things he missed. “Every time I go through I’m like, I didn’t notice that one! And in each case, these are discoveries that we can understand and verify,” he says.
Pretty vague and not really quantifiable. You would think an article making a bold claim would contain more than a single, hand-wavy quote from an actual scientist.
Why? What purpose would quotes serve better than a paper with numbers and code? Just seems like nitpicking here. The article could have gone without a single quote (or had several more) and it wouldn't really change anything. And that quote is not really vague in the context of the article.
What is interesting is that HN seems to have reached a crescendo of AI fanboi posts. Yet if you step outside the bubble the Microsoft and Nvidia CEOs are begging people to actually like AI, Dell's come out and said that people don't want AI, and forums are littered with people complaining about negative consequences of AI. Go figure.
Taking CV-filler from 80% to 95% of published academic work is yet another revolutionary breakthrough on the road to superintelligence.
Is it cynical to believe this is already true and has been forever?
Is it naive to hope that when AI can do this work, we will all admit that much of the work was never worth doing in the first place, our academic institutions are broken, and new incentives are sorely needed?
I’m reminded of a chapter in Abundance where Ezra Klein notes how successful (NIH?) grant awardees are getting older over time, nobody will take risks on young scientists, and everyone is spending more of their time churning out bureaucratic compliance than doing science.
Funny you say that.