Comment on d wha

<- View Parent
lambdabeta@lemmy.ca ⁨1⁩ ⁨week⁩ ago

Yeah, if words were actually encoded as 1-hot vectors this would be pretty trivial, but the rest of LLM training would be somewhere between infeasible and impossible. The actual embedding vectors obscure spelling even more.

Side note: last time I checked, current embedding vectors were approximately 40 dimensional… Has that gone up significantly in the last couple of years?

source
Sort:hotnewtop