Meta Doubles Training Efficiency on Its LLM-Scale Ads Model

GEM sits at the awkward intersection of recommendation systems and language models, where the ragged tensor shapes typical of recsys features break the dense-attention assumptions that LLM training stacks are built around. Meta’s answer was hardware and software co-design: a custom recommendation kernel library featuring Jagged Flash Attention and mixed ultra-low-precision training tuned specifically to those workloads, roughly doubling compute efficiency. The generalisable lesson for anyone training on non-uniform sequence data is that generic attention kernels leave close to half the available compute unclaimed once your inputs stop looking like text.