Comment on What is a good eli5 analogy for GenAI not "knowing" what they say?
GamingChairModel@lemmy.world 6 months agoThe idea that these models are just stochastic parrots that only probabilisticly repeat their training data isn’t correct
I would argue that it is quite obviously correct, but that the interesting question is whether humans are in the same category (I would argue yes).
HorseRabbit@lemmy.sdf.org 6 months ago
People sometimes act like the models can only reproduce their training data, which is what I’m saying is wrong. They do generalise.
During training the models are trained to predict the next word, but after training the network is always effectively interpolating between the training examples it has memorised. But this interpolation doesn’t happen in text space but in a very high dimensional abstract semantic representation space, a ‘concept space’.
Now imagine that you have memorised two paragraphs that occupy two points in concept space. And then you interpolate between them. This gives you a new point, potentially unseen during training, a new concept, that is in some ways analogous to the two paragraphs you memorised, but still fundamentally different, and potentially novel.