Language & LLMs

What Is an Autoregressive Model?

An autoregressive model produces a sequence one element at a time, where each new element depends on the previously generated ones. In language modeling, this means predicting each token from all prior tokens. Most text-generation models, including GPT, are autoregressive.