Language & LLMs

What Is a Contextual Embedding?

A contextual embedding assigns a word a vector that depends on its surrounding context, so the same word can have different representations in different sentences. This contrasts with static embeddings like word2vec, which give each word a single fixed vector. Models such as BERT and GPT produce contextual embeddings.