CS336: Language Modeling from Scratch
159 points
3 hours ago
| 7 comments
| cs336.stanford.edu
| HN
meken
2 hours ago
[-]
I have fond memories of cs224d [1] taught by richardsocher. It’s a bit dated at this point as it was created in the pre-transformer era, but it was a very cool introduction to applying deep learning to nlp at the time.

[1] https://cs224d.stanford.edu

reply
egl2020
1 hour ago
[-]
Similar thoughts here. That was when I realized the potential of the Internet: I didn't have to be a grad student at a tier 1 research university to learn about the frontier.
reply
skerit
1 hour ago
[-]
> GPU compute for self-study

Those suggestions they make for a B200 start at $4.99 an hour.

Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai

reply
grahameb
15 minutes ago
[-]
It seems strange that the required resources aren't provided by the educational institution?
reply
ReptileMan
2 minutes ago
[-]
Two schools of thought - people are paying 100K per year, we should provide everything. Second is - they are paying 100K per year, do you think they will care for couple of hundred more.
reply
root-parent
1 hour ago
[-]
You dont even need a GPU to train your own LLM.
reply
flakiness
36 minutes ago
[-]
I beliee these are affordable enough for the intended audience (which is Stanford undergrad/master)
reply
sonabinu
41 minutes ago
[-]
I brought a group together to do this class using the YouTube videos and course materials available online. It is challenging but rewarding. We tackled it one lecture video per week. Started with over 30 learners and by last session we were down to 8.
reply
airstrike
1 hour ago
[-]
I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in
reply
danbrooks
18 minutes ago
[-]
I'd be interested in joining a discord server.

Would be great to have a community to discuss the material - even if folks can't commit to the full course.

reply
storus
2 hours ago
[-]
Thanks for releasing this again! What are this year's changes to prior offerings?
reply
dominotw
19 minutes ago
[-]
i recently started reading "build reasoning model from scratch" then i realized that i am not really interested in building part and just want to understand theory and practice behind it.

A want like a casual lesswrong style from ground up explanation.

reply
tmule
1 hour ago
[-]
Are video lectures available online?
reply
Bilal_io
1 hour ago
[-]
reply
aerohit
1 hour ago
[-]
reply
mindcrime
1 hour ago
[-]
reply