Comment on Squiggly Boie

<- View Parent
NotANumber@lemmy.dbzer0.com ⁨3⁩ ⁨days⁩ ago

Yes genetic algorithms are something different. Though they are used sometimes in training or arcitecting NNs, but not at the scale of modern LLMs.

Fyi you can have all or nothing outputs from a perceptron or other network. It all depends on the activation function. Most LLMs don’t use that kind of activation function, but it is possible. Have you heard of bitnet? They use only one of three states for the neuron output in an LLM. It’s interesting stuff.

source
Sort:hotnewtop