Infrastructure & Agents

What Is Model Parallelism?

Model parallelism divides a model itself across several devices so that each holds only part of the network. This lets teams train models whose parameters exceed the memory of any single GPU.