Software developer here, the more I learn about neural networks, the more they seem like very convoluted statistics. They also just a simplified form of neurons, and thus I advise against overhumanization, even if they’re called “neurons” and/or Alex.
Comment on answer = sum(n) / len(n)
General_Effort@lemmy.world 2 months ago
Neural nets, including LLMs, have almost nothing to do with statistics. There are many different methods in Machine Learning. Many of them are applied statistics, but neural nets are not. If you have any ideas about how statistics are at the bottom of LLMs, you are probably thinking about some other ML technique. One that has nothing to do with LLMs.
ZILtoid1991@lemmy.world 2 months ago
General_Effort@lemmy.world 2 months ago
he more I learn about neural networks, the more they seem like very convoluted statistics
How so?
Kethal@lemmy.world 2 months ago
Hahaha. People are great. Image
Image
General_Effort@lemmy.world 2 months ago
That’s where the almost comes in. Unfortunately, there are many traps for the unwary stochastic parrot.
Training a neural net can be seen as a generalized regression analysis. But that’s not where it comes from. Inspiration comes mainly from biology, and also from physics. It’s not a result of developing better statistics. Training algorithms, like Backprop, were developed for the purpose. It’s not something that the pioneers could look up in a stats textbook. This is why the terminology is different. Where the same terms are used, they don’t mean quite the same thing, unfortunately.
Many developments crucial for LLMs have no counterpart in statistics, like fine-tuning, RLHF, or self-attention. Conversely, what you typically want from a regression - such as neatly interpretable parameters with error bars - is conspicuously absent in ANNs.
Any ideas you have formed about LLMs, based on the understanding that they are just statistics, are very likely wrong.
Kethal@lemmy.world 2 months ago
“such as neatly interpretable parameters”
Hahaha, hahahahahaha.
Hahahahaha.