Activation Function
In plain terms
A function adding non-linearity to a neural network.
Going deeper
An activation function transforms a neuron's output non-linearly, letting networks learn complex patterns beyond straight lines. Without it, layers collapse to linear math.
Under the hood
Choices (ReLU, sigmoid, tanh, GELU) affect training; ReLU dominates for hidden layers.
Examples
ReLU, sigmoid, tanh, softmax, neural network layers, non-linearity.
Think of it like...
Like a dimmer switch deciding how strongly each neuron 'fires' based on its input.