Developers Are Writing Their Own C Inference Engines Again

Three signals landed the same day. LocalAI published on why it writes its own C and C++ inference engines rather than adopting a general runtime. Antirez’s ds4, a dedicated local engine for DeepSeek 4 Flash and Pro, trended in C. And AirLLM, which runs 70B inference on a single 4GB GPU, took around 819 stars. The common cause is architectural: today’s frontier open models are sparse mixture-of-experts, and runtimes built for dense weights handle them badly — routing, expert caching and I/O scheduling are the hot path now, not matrix multiplication. When engineers of that calibre bypass established runtimes, the runtimes are the story.