Cross Entropy
In plain terms
A loss function measuring difference between probability distributions.
Going deeper
Cross-entropy loss penalizes the gap between predicted probabilities and true labels, standard for classification. It heavily punishes confident wrong predictions.
Under the hood
Pairs with softmax/sigmoid outputs; lower values mean predictions match the true distribution better.
Examples
Classification training, LLMs, softmax output, log loss, neural networks.
Think of it like...
Like grading confident wrong answers much more harshly than hesitant ones.