Model Training as Code
36 points
3 days ago
| 3 comments
| aleph-alpha.com
| HN
delichon
5 hours ago
[-]
Some good stuff here from Dwarkesh around mashing up training and inference:

https://youtu.be/20p5-kQXF_Q?is=72ImTNxkOEKmOXQ9

He predicts this kind of model factory will become central to organizational learning and operations. Updating and upgrading the model stack becomes the core staff function.

reply
faangguyindia
2 hours ago
[-]
Interesting points made in the video.

But models did not become good at coding just because coding is replayable. It’s because there are countless repos, issues, Stack Overflow threads, and Reddit posts/comments/questions where a solution is clearly marked as “solved” or “that helped,” and AI can learn from that feedback.

Being replayable does play a role because a solution can be tested against a compiler, and the resulting errors or lack of errors/warnings can reveal whether it worked.

This becomes much harder in fields like fitness, where changes take much longer and cause and effect relationships are not straightforward to establish.

Your muscle gain increased but was it because you increased protein intake? Or was it because you started eating more carbs, which added more energy to the system?

Once protein needs are already met, calories may become the limiting factor. In that case, the additional gains may come primarily from increased calorie intake rather than the higher protein intake itself.

AI is bad at fitness, evidently.

Many people forget, conversation with a model also generates training data. This is how your problems, algorithms, solutions end up in training data and end up right at your competitors without your competitor trying to actively steal your code.

I simply do not expose core algorithms which improve my product to AI agents.

reply
jaggederest
4 hours ago
[-]
I think this is an interesting thing that will happen once the rate of change slows down a little bit - imagine a world where there's more or less a couple base models and everyone trains on top of them, and the bitter lesson is defunct just via sheer physics (maybe we have the best models we can physically run in reasonable energy density substrates, or something), then it becomes "your personal model" with your overlay, training, or feedback on top.
reply
SpyCoder77
5 hours ago
[-]
What is this "aleph" thing in names now? First aleph neuro, and now aleph alpha.
reply
verelo
5 hours ago
[-]
I'm glad you're asking because I've seen it too and don't get it either. I assumed initially it was alpha as a typo, then I Googled it and got even more confused.
reply
boothby
4 hours ago
[-]
First letter of the Hebrew alphabet, used by mathematicians to denote infinities.
reply
verelo
4 hours ago
[-]
That's what Google told me, but i still don't see how it links to this?
reply
UltraSane
2 hours ago
[-]
It is just vibes man. It sounds cool, nothing more.
reply
akoboldfrying
2 hours ago
[-]
It doesn't -- it's marketing, much like adding "Labs" to the end of your company's name. Its association with infinity makes the company sound cooler to potential customers, many of whom are software engineers who consciously or unconsciously view pure mathematics as a prestige "final form" of their own logic-focused mental ability.
reply
random3
5 hours ago
[-]
> TL;DR: Model training has grown complex

So they’ve built Savanah - a workflow engine because the existing zoo of hundreds of workflow engines didn’t cut it :)

reply