https://proceedings.mlr.press/v137/kavalerov20a/kavalerov20a...
It turns out 2 classes is special. Better to add the classes as side information rather than try to make it part of the main objective.
In practice, when it comes down to code, even without higher-level libraries, it is surprisingly simple, concise and intuitive.
Most of the math elements used have quite straightforward properties and utility, but of course if you combine them all together into big expressions with lots of single-character variables, it's really hard to understand for everyone. You kind of need to learn to squint your eyes and understand the basic building-blocks that the maths represent, but that shouldn't be necessary if it wasn't obfuscated like this.
But my experience as a mathematician tells me another part of that story.
Certain fields are much more used to consuming (and producing) visual noise in their notation!
Some fields have even superfluous parts in their definitions and keep them around out of tradition.
It's just as with code: Not everyone values writing readable code highly. Some are fine with 200 line function bodies.
And refactoring mathematics is even harder: There's no single codebase and the old papers don't disappear.
I think people forget why math is used. I'm always a little surprised that programmers don't see this because the languages are being used for the same reasons. Precision. They're terrible languages to communicate something like this conversation but then again English is a terrible way to communicate highly abstract concepts.
On the other hand, I've definitely seen people use math to make their works seem more important (definitely in some ML) I think I more frequently see it just being copy pasted (like every diffusion paper ever). I think that is probably superfluous, though it's definitely debatable and I'm absolutely certain these use cases aren't for flexing lol.
Maybe for Von Neumann math was simple...
They threw in all the complex math to the paper. I could not initially understand it at all despite inventing the damn algorithm!
Having said that, picking it apart and taking a little time with it, it actually wasn't that hard - but it sure looked scary and incomprehensible at first!
Think of it this way
You don't need math to train a good model but you need math to know why your model is wrong.
Also, the idea of using a paper to signal intelligence is kinda silly. Papers aren't being written for the general public, papers are the communication between scientists. Who are they impressing? Each other? The others who are going to call them out if they write bullshit or make arguments convoluted? I don't buy that. But maybe because I'm a researcher. But I also don't think I need to use math to look smart, my PhD and publication record do a good enough job of that on their own. I don't even need it to flex to other researchers. The math in my papers is because it is just easier to communicate. I'm sure there's concepts that you find easier to understand by reading code than by using English. Same thing. Math and programming are great languages when you need high precision and when being pedantic is essential. Math is used because it is the best way to communicate, not as a flex. We flex on each other by showing how our ideas are the best. You can't do that if the other person doesn't understand you.
@staticelf and anyone else that feels that way:
That feeling is normal in the beginning. Basically your first year of a PhD is spent going "what the fuck does any of this mean?!?!" It's rough. But also normal. You're working at the bounds of human knowledge and papers are written in the context of other papers. It's hard to jump in because it is like jumping into the middle of a decades (or longer) conversation. If you didn't feel lost then the conversation probably wasn't that complicated and we'd probably have solved those problems much earlier. So you sit down and read a lot of papers to get context to that conversation.
My point is, don't put yourself down. The hill you need to climb looks steeper than it is. Unfortunately it is also hard to track your progress so you tend to feel like it's continually out of reach until it suddenly isn't. (It's also hard because everyone feels like an imposter and many are afraid to admit not knowing. But the whole job is about not knowing lol) Probably the most important skill in a PhD is persistence. I doubt you're too stupid. I'm sure you can look back and see that you've done things you or other people are really impressed with. Things that looked like giant mountains to climb but looking back don't seem so giant anymore. We'd get nowhere in life if we didn't try to do things we initially thought were too hard. Truth is you never know till you try. I'm not going to say it's easy (it isn't), but that it isn't insurmountable. You can't compare yourself against others who have years of training. Instead look at them and see that that's where this training can take you. But you can't get there if you don't try.
Also there are libraries that abstract away most if not all the things, so you don't have to know everything
This statement is not true, there are counter examples I encountered in my university studies but I would say that intuition will get you very far. Einstein was able to come up with special theory of relativity by just manipulating mental models after all. Only when he tried to generalize it, that’s when he hit the limit of the claim I learned in school.
That being said after abandoning intuition, relying on pure mathematical reasoning drives you to a desired place and from there you usually can reason about the theorem in an intuitive way again.
Math in this paper is not that hard to learn, you just need someone to present you the key idea.
> behind every hard formula lies an intuitive explanation
I agree you're definitely right that a lot of math (if not most) is really unintuitive. But I think I still like the sentiment behind that idea. Maybe it changes with translation, but I feel like that equations make a lot more sense when I break them down and think about what the symbols are doing and looking at their relationships with one another compared to when just looking at them as symbols to manipulate. Like seeing the form F=ma as more than mass and acceleration and but how in structure it is similar to F=-kx. Getting there isn't easy but once you do it is much more intuitive than it was before.
I hope anyone who is unsure will read your comment and at least try to follow it for a while.
Last time I used a GAN was in 2015, still interesting to see a post about GANs now and then.
I think the weirdest thing in ML has always been acting like there's an objectively better model and no context is needed.
Yes you can just concentrate on the latest models but if you want a better grounding in the field some understanding of the past is important. In particular reusing ideas from the past in a new way and/or with better software/hardware/datasets is a common source of new developments!
Though if you can rephrase the problem into a diffusion it seems to be prefered these days. (Less prone to mode collapse)
Gan is famously used for generative usecases, but has wide uses for creating useful latent spaces with limited data, and show up in few-shot-learning-papers. (Im actually not that up to speed on the state of art in few-shot so mabie they have something clever that replace it)
GANs were fun though. :)