Moonshot Open-Sources Its Kimi Delta Attention CUDA Kernels

Moonshot AI published FlashKDA, MIT-licensed CUTLASS kernels implementing Kimi Delta Attention — the gated, recurrent-state attention variant behind its Kimi models. It ships a forward-pass flash_kda.fwd() taking queries, keys, values and gates, requires SM90 or newer with CUDA 12.9 and PyTorch 2.4, and includes separate benchmark notes for H20 and GB200. Releasing kernels rather than weights is the quieter form of open source and often the more useful one: an architecture nobody can run efficiently stays a paper, and a kernel is what turns it into something others can build on.