Was going to say much the same. I recall one story about a genetic algorithm to make an oscillator with the fewest possible components, and it successfully did so by surprising the humans with a single wire, i.e. an antenna picking up nearby stray RF.

This too: https://en.wikipedia.org/wiki/Evolvable_hardware

Starting with: https://sci-hub.ru/storage/moscow/4324/11d145b2c2c3ab320f70b...

> our common expectation

I think you're going too far with this. Most people understand scientific theories to be an approximation. F=ma is approximately true, in the sense that it's only accurate within the newtonian regime and each of those terms includes so many asterisks that you will only ever measure it approximately.

The latter is the jokes about the physicists "assuming a perfectly spherical cow."

In fact that's kinda the whole point of the "unreasonable effectiveness of mathematics" essay. It is unreasonable that mathematical approximations are so good at describing our world.

I often think this about medicine and the human body. We want to believe that our bodies are some miraculous well oiled machine. But it often seems that it’s a barely held together bag of mess.

Would it be sad? If it’s gnarly and it solves the problem, as an end user I don’t really care. The only people who lose are the mathematical purists

This has been on my mind lately! Especially in light of the many incomprehensible but machine-checkable proofs we've been hearing about.

Occam's Razor is a useful heuristic, but it biases us towards simpler explanations.

The "common expectation" I think, misses the point. The idea isn't that fundamental theories are simple or elegant (quantum physics equations are pretty darn ugly), it's that, given the choice between a more complicated and a more simple theory, generally the simplest one is the most accurate choice.

That is very unlikely due to Solomonoff induction...

>I wonder if our common expectation that true theories somehow had to be beautiful and elegant is going to survive the coming century.

That's the layman's idea of physics theories. They are beautiful and elegant only on the surface, that's why they're technically models and approximations of the real world. The standard model renormalization techniques are a mess of patches and ad-hoc heuristics, pretty far from the "this lagrangian literally contains all physics". Generally you just _ignore_ higher order terms and just call it a day. The famous E=mc^2 it's just the first term of a Taylor expansion. The beautiful form of physics it's what you would call "good enough" and often just a pedagogical tool.

It would be really cool. We already know everything at the lowest levels is a probability cloud. There’s beauty and contentment in not really being able to nail anything down for eternity…

Does the nature look like a horrible mess to you?

How would nature be best described as a horrible mess of impossible equations? They would be best described as elegant and beautiful no?

I think your point is more that we might be able to initially describe complex phenomena as messy, horrible complex equations, that doesn’t mean we shouldn’t work to simplify them and make them more understandable to us.

I’m reminded of lawyer Damien Riehl’s (performative) reaction to the Sam Smith infringement decision, back in 2019/2020. He and programmer Noah Rubin algorithmically generated every possible melody (within a certain combinatorial space, in MIDI format as I recall), and purported to release them under CC-0 license [0]. He went on to attract some attention and explain his argument at a regional TEDx event [1].

I seem to recall legal commentators reacting with an eyeroll—apparently judges split much finer hairs than these for a living—but it was a cute stunt.

[0] https://allthemusic.info/

[1] https://m.youtube.com/watch?v=sJtm0MoOgiU and https://www.the-independent.com/tech/music-copyright-algorit...

Wouldn't work. Judges would not treat the AI generated designs as prior art without proof of human involvement (above and beyond entering the prompt).

Machine learning layer cake with some brute force crumbs.

It's marketing bullshit. For one, it's like proving a negative; you can't prove to me that humans couldn't have imagined it. Second, humans have already imagined quite a lot of crazy stuff...

It is a problem because people will talk about what AI can do implying that an LLM can do that thing, making it seem like a pure LLM can do almost anything. On the other hand people will say AI will never be able to do X because an LLM can’t do that thing well natively. AI has become too vague of a term to be useful.

I wish I could wave a magic wand and just make the word "AI" go away. It has no actual meaning. It could mean anything from your opponent in Mario Kart to Stable Diffusion.

"AI" is a term cursed by cool sci-fi implications. It makes it a kick ass marketing term because most people are going to have some familiarity with sci-fi AI and "X media predicted Y technology" is a pretty widespread belief for a lot of values of X (star trek, Hitchhikers Guide to the Galaxy, Arthur C. Clarke) and Y (internet, cell phones, VR). If you want to tell someone we're making big strides in something, linking it into some popsci understanding of sci-fi being the great predictor of human achievement is low effort and high impact for quite a few people.

People aren't trying to communicate accurately if their first priority is getting you excited about the thing!

I have been practicing saying ML for traditional machine learning and LLMs for LLMs for just this reason. Trying not to say AI anymore. Too ambiguous. Sometimes I'm talking about game AI even, I'll try to use shorthand for whatever algorithm I think the AI is using (often I'll talk about its flowchart, though not always sure it's literally using that under the hood).

Just as more successful machine learning fields distanced themselves from the term during the AI winter, I suppose we will (and perhaps are?) be seeing them adopt it again, now that we are in an "AI summer".

Recently I heard some people conflate procedural generation and generative AI and I had to explain why there isn't some legal or ethical issue with what breaks down to essentially scattering some points.

It's really getting annoying having to have these conversations.

Reinforcement learning can solve a Rubik’s Cube. A LLM that hasn’t been trained to solve a Rubik’s Cube can not.

> AI can do a looot of things

AI is not a real thing or a natural kind but a perspective. Whether something qualifies as "AI" or not cannot be decided by the objective features of the thing. Ergo, it can be defined at the author's pleasure.

> conflating established and morally neutral activities in ML

LLMs are no more or less morally neutral than other ML techniques.

No. I'm really not fan of Python but you're not going to parse JSON at assembler level. You need to choose right level of indirection for each task. And LLMs are good at more than Python and Rust. I'm using LLMs to write programs in my own language that is compiled by a very fringe language and it happily does everything from scripting on ESP32 to audio plugins and 3D gfx on desktop.

I’m intrigued by the question but I do think it has some worrying implications for portability.

I was going to post this as well, its delightful to see that other people enjoy it since it was really mind-blowing when I read it.

It's interesting since I saw another comment near yours that raised the question of robustness of the lab-grown design, which I thought was kind of the most fascinating part of the damninteresting article was the revelation that the evolved programs were inseparable from the single physical FPGA used in the training. Since this RFIC training model employs a simulator, do you suppose that the quirks of the physical hardware on which the simulator runs are sufficiently isolated from the training such that a pair of designs would behave similarly when the simulator was run on distinct hardware? And I guess the even more obvious question is whether a design evolved on a simulator would have any hope of behaving as expected in physical hardware?

My hunch about the latter is no, although it still seems like an interesting study, and I often find myself thinking that really understanding what was going on with the FPGAs might be a prerequisite for really understanding how to master reinforcement learning.

Anyway I'm glad you posted this and if you have any other favorites related to this domain send them my way!

Yes. An example of a species so specialized and optimized that it can no longer adapt. Also, an example of POSIWID.

> but i didn't find them addressing it explicitly in the text

Yes, this is exactly what bothers me about this article and about a few similar articles published in the past, that they do not contain any evidence that their claims about the usefulness of AI in design are true.

In TFA it says that the role of AI is replacing the electromagnetic simulator in the optimization process, by guessing the behavior of the structure, which is many orders of magnitude faster than a simulation.

This sounds plausible, but in order to believe this I would want to see the differences between AI guesses and real measurements, in the case of structures with geometries that are very different from those used in the training of the AI.

Also I would want to see exactly with which simulators they have compared the speed of the AI model.

There are various simulation approaches for electromagnetic fields and electronic circuits, that can trade-off accuracy for speed, so I am not convinced that AI inference takes necessarily much less time than some faster low-accuracy methods of simulation, which would still be more accurate and more reliable than AI guesses.

I came to mention genetic antennae as well!

Since you beat me to it, I'll add something that relates relates you were saying on "realizing some pretty simple... ideas".

I think a big plus of computer aided design like this is "innovization"[1]. Somewhat awkward term. But, a system like this leading one to deeper understanding of a particular process is the general idea. It's a fun feeling in practice.

[1]: https://dl.acm.org/doi/10.1145/1143997.1144266

Yeah it's a hype slop piece

Does it need to be magical to be interesting or useful?

And what about all the movie plots ruined by the existence of cell phones?

The AI in the article isn't an LLM.

Read the article.

Yep, can't wait until everything is free and costs nothing to generate content. Free hosting and electricity will be super sweet too. Won't need admins or even the Internet. Everything I want will just be free for me because I don't think anything has value.

>To arrive at the “sweet spot” where all these different parameters are balanced into optimal harmony, designers will typically lay out several different versions of the circuit, using intuitions and methods they have picked up in their years of training.

I kind of thought the real success is when the designer comes up with key things that are well beyond their training or any training that could have been done up until that time. Based on their years of experience living in an environment where training is table stakes but that's not the thing that's relied upon the most in the end.

With LLMs it seems like odds are that a concept which is statistically insignificant in the training set may surface in place of a truly novel solution, effectively displacing the real breakthroughs that actually go beyond trainable performance.

In a way that decision-makers can not tell the difference, and that could be the worst part.

> telling people it's all "black magic" and "wizardry" (and worst of all, saying "even RF engineers don't understand RF") makes it seem like it's not worth studying.

I think the opposite is true. It being advertised as difficult to understand is one of the reasons I personally decided to study RF Engineering. The prospect of learning something so challenging pulled me in. The Smith Chart helped.

Unexpected Star Wars! A surprise to be sure, but a welcome one :)

How does your consciousness work?

Then why can't these constraints be encoded into the selection/scoring function ?

This is analogies to finding a new prime number by brute force using existing maths, rather than inventing new maths to get there.

The AI in this case didn't create a novel technology- it merely used the existing technology without basing the new design on a previous one. The whole "human couldn't come up with it" is because the possible design space is so large, there's no reason a human would start where the AI did.

The thing the AI did better than humans was brute forcing a solution faster. Still a very handy thing to have, but it isn't "creating" in the sense that it invented new materials or fabrication processes or anything novel.

Have you read the article? The creative element came from the researchers:

> In our new approach, the architecture begins essentially from nothing and is progressively assembled through successive iterations. The system explores the design space by generating myriad candidate circuit combinations and mapping the resulting performance trade-offs as it navigates this landscape. Because the process is not biased by prior human design choices, it can produce completely novel circuit topologies that look markedly different from those created by human designers.

> I read on HN that AI's can just interpolate the data they have seen in training

No. That can be said about LLMs, but not about all forms of AI. The technique used is not a LLM.

Sadly we've bastardized the term AI that, if it ever meant anything, it's meaningless now. The currently most voted thread in this post discuses the topic.

In my experience, if you tell them to research the web to see if their idea has been pursued before, you can get them to keep proposing new things until something is sufficiently new, even if it's a new interpolation between existing concepts, that it's effectively an original idea.

This is wrong - the training data is necessary but insufficient. There are a lot of other parts of the architectures used that add a lot of value - otherwise Markov chains would be all you need. There are layers upon layers with non linear activation functions, learned residuals, etc. They still absolutely must interpolate but the space they interpolate through is much more complex than the training data, and they can definitely create things not in their training data. What they can not do is wander outside their non linear parameter space’s convex hull. But this is a really permissive constraint on what they can do “creatively.” People generally under estimate the advantage the architectures confer on that constraint. This is why there was a step function change in expressive power as the architectures (attention, self attention, transformers, diffusions, others) evolved given the same training data. Generally though I challenge you to define “creative” in a way that is precise enough to measure and isn’t self referential or refer to concepts ill defined.

The key tho is can they solve problems not easily solved before with prior techniques. Further can they identify problems not readily presented. Then identify novel solutions. Etc. The answer is emphatically yes they can. These features don’t have to literally exist in their training data, but the supporting highly convoluted network of associations of all their training data does have to in some complex space allow for it to produce these answers. It’s not the same as they’re stochastic parrots at all.

Are they creative? No, because they don’t have awareness. My personal imprecise definition of creative requires both self and awareness as well as free will. There is no driving awareness in all AI architectures, it all derives from extrinsic impetus. Creativity is derived, IMO, from a layer of our minds that is not readily assessed or measured and is only indirectly expressed through language, art, and music. Hence it is not directly trainable and therefore a learning model can’t learn it by reinforcement. It can learn the proxies, but the proxies are not, as we all deeply know, the same as our experienced awareness. We are not our words, our art, our music. We try hard to bridge it, but it’s impossible and you and I know this to be true from experience. In fact we can not even examine our own awareness because it’s not directly observable or possible for us to directly reason about. This is core to a lot of philosophy, especially mid and far eastern philosophy of the mind, the self, the five aggregates of Buddhism, etc. Psychology points at it, and modern psychology avoids it because it’s practically difficult for outcome oriented treatments.