Language & LLMs

What Are Logits?

Logits are the raw numerical scores a neural network produces for each possible output before normalization. In language models, a softmax function converts logits into a probability distribution over the vocabulary. Decoding strategies then use these probabilities to choose the next token.