Comment on no one can understand AI
mrchampion@lemmy.world 20 hours agoGood old ReLU. I’ve heard that CNN’s perform better when using ReLU as the activation func, and that’s probably why; ReLU acts as a filter on the image’s features.
Comment on no one can understand AI
mrchampion@lemmy.world 20 hours agoGood old ReLU. I’ve heard that CNN’s perform better when using ReLU as the activation func, and that’s probably why; ReLU acts as a filter on the image’s features.
Gladaed@feddit.org 20 hours ago
Nah. Without a nonlinearity you just get a linear combination of inputs instead of the output of a deep neural network. ReLU or Ramp is just the simplest possible non linearity. Using a simple function can enable using deeper networks yielding even better performance.
It’s actually somewhat of a headache, numerically. Works well enough tho.