Infrastructure & Agents

What Is Pipeline Parallelism?

Pipeline parallelism places consecutive layers of a model on separate devices, passing activations from one stage to the next. Batches are split into micro-batches so devices can work concurrently and reduce idle time.