The 90/10 Split: Small Models Are Absorbing the Frontier's Work

Q: How many parameters does a model actually need to be useful now?

Far fewer active parameters than the total parameter counts in headlines suggest, and the gap between those two numbers is the whole story. Mixture-of-experts models route each token through a small subset of their weights, so a model can have enormous total capacity while only activating a fraction of it per token. Upstage's Solar Open 2 has 250 billion total parameters but activates roughly 15 billion per token, and its published SWE-Bench Verified score is 70.4. Poolside's Laguna-S-2.1 has 118 billion total and activates around 8 billion, with a published Terminal-Bench 2.1 result of 70.2%. Both are vendor-reported figures rather than independently reproduced ones, but the direction is consistent across several independent releases: capability that required frontier-scale dense compute two years ago is now reachable at single-digit-billions of active parameters for well-scoped tasks. Serving cost tracks active parameters far more closely than it tracks the total, which is why the active number is the one worth watching.

Q: Can you really fine-tune a competitive model for a few hundred dollars?

For a narrow, well-specified task with a clean reward signal, the evidence suggests yes, with important caveats. Fermisense reported reinforcement fine-tuning a 9-billion-parameter open model using GRPO for roughly $500 of GPU time — about 1,000 optimization steps over three and a half days on two workstation-class GPUs — and reported that it outperformed five frontier configurations on a catalog-review workflow at dramatically lower per-unit cost. The critical caveats are that the workflow was simulated rather than drawn from production, the evaluation has not been independently replicated, and the company publishing the result has a commercial interest in it. The claim that generalizes is narrow but still significant: the capital requirement for producing a task-specific specialist has fallen to roughly the price of two GPUs and a long weekend. The claim that does not generalize is 'open models beat frontier models' — that is one simulated task, not a law.

Q: What is GRPO and why does it keep appearing in these results?

Group Relative Policy Optimization is a reinforcement learning method for language models that estimates the advantage of a response by comparing it against a group of other sampled responses to the same prompt, rather than by training a separate value network to predict expected reward. Removing the value model matters practically: it roughly halves the memory needed during training and eliminates a component that is itself difficult to train stably. That memory reduction is what brings the technique within reach of two-GPU setups instead of clusters. GRPO shows up repeatedly in small-specialist results because it fits the shape of the problem — narrow tasks usually have a programmatic or rule-based reward signal available, which is exactly the setting where reinforcement learning works well and where supervised fine-tuning on a fixed dataset tends to plateau.

Q: Does this mean frontier models are becoming less important?

No, and the cascade framing actually makes the opposite case. In a 90/10 split the frontier model is what handles the requests that genuinely matter and that nothing cheaper can do — it is the escalation path, and the quality of the whole system is bounded by it on exactly the hard cases where quality is most visible. What changes is the volume of traffic sent to it and therefore the shape of the market. Frontier capability becomes a smaller fraction of tokens served and a larger fraction of the value delivered per token. There is also a dependency worth naming: the small specialists in these results are frequently distilled or fine-tuned from larger models, and Microsoft's security model is explicitly derived from a mid-training checkpoint of a larger internal model. The cheap tier is downstream of continued frontier investment rather than a replacement for it.

Q: How do I decide what to route where?

Start by measuring the difficulty distribution of your actual traffic before building anything, because the entire economic case depends on the easy majority being real rather than assumed. Sample production requests, have your current model handle them, and label which ones a substantially smaller model could also have handled correctly. If that fraction is not large, stop — a cascade will not pay for itself. If it is large, the next decision is what signal triggers escalation, and the options in rough order of implementation cost are: explicit task-type routing based on request metadata, a cheap classifier trained on your difficulty labels, the small model's own token-level confidence or self-assessment, and verification of the small model's output where the task admits a programmatic check. Verification is the strongest signal when it is available, because it converts an uncertain prediction problem into a deterministic one, which is precisely why the pattern first matured in domains like code and security where outputs can be tested.

Two results landed within a day of each other this week, from parties with nothing in common, and they point in the same direction.

Microsoft released a cybersecurity model with 5 billion active parameters and said plainly that it was built to handle up to 90% of the tasks in its vulnerability-analysis pipeline, so that a frontier model would only be called for the remaining 10%. Separately, a small company reported that roughly $500 of reinforcement fine-tuning on a 9-billion-parameter open model beat five frontier configurations on a narrow catalog-review task at a fraction of the per-unit cost.

Neither result is, on its own, proof of very much. Together with a run of open-weight releases whose active parameter counts keep falling while their benchmark scores keep rising, they describe a change in how inference gets bought: the default is shifting from one large model for everything to a small specialist with a large escalation path.

Executive summary

Microsoft’s MAI-Cyber-1-Flash is a 137-billion-parameter mixture-of-experts model that activates 5 billion parameters per token, with a 256,000-token context window. Microsoft states it was designed to absorb up to 90% of tasks so that costlier models handle only the hardest 10%.
The reported result of that split is roughly 96% on the CyberGym benchmark at about half the cost of the previous configuration — a cost reduction achieved by architecture rather than by a cheaper model.
Fermisense reported that a $500 GRPO fine-tune of a 9B open model beat frontier baselines on a simulated catalog-review workflow. The workflow was simulated and the result has not been independently replicated.
Open-weight releases are converging on low active-parameter counts: Solar Open 2 at 15B active of 250B total, Laguna-S-2.1 at roughly 8B active of 118B total, both reporting benchmark results in the range that required far larger models a year ago.
Serving cost tracks active parameters, not total parameters. The headline number in a model announcement is increasingly not the number that determines your bill.
The hard part of a cascade is the escalation decision. Everything else is plumbing.

What Microsoft actually built

MAI-Cyber-1-Flash is a sparse mixture-of-experts transformer with 137 billion total parameters, 5 billion active per token, and a 256,000-token context window. It is a cybersecurity fine-tune of MAI-Code-1-Flash, which was itself developed from a mid-training checkpoint of a larger internal reasoning model. It powers MDASH, Microsoft’s harness for identifying and remediating software vulnerabilities.

The reported headline is that MDASH, running this model alongside a frontier model for escalation, reaches about 96% on the CyberGym benchmark at roughly half the cost of Microsoft’s previous MDASH configuration.

The number worth attention is not 96%. It is the 90/10 split, stated as a design goal:

   Incoming task
        │
        ▼
 ┌──────────────────┐
 │ MAI-Cyber-1-Flash│   5B active params
 │   137B total     │   handles ~90% of tasks
 └──────────────────┘
        │
        ├── resolved ──────────────►  done  (cheap)
        │
        └── escalate (~10%)
                 │
                 ▼
        ┌──────────────────┐
        │  Frontier model  │   expensive, reserved
        └──────────────────┘   for the hard tail

This is a classifier cascade, a pattern that predates the current era by decades. What is new is a hyperscaler publishing it as the product architecture, with the cost saving as the headline claim rather than as an internal efficiency note.

It works because task difficulty in real workloads is rarely uniform. In vulnerability analysis, as in most engineering work, a large majority of cases are recognizable patterns and a small minority require genuine reasoning about unfamiliar code. Paying frontier prices on the whole distribution means overpaying on nearly all of it.

The security domain also happens to be unusually well suited to the pattern, for a reason worth extracting: you can often verify the answer. A proposed vulnerability can be tested against a proof of concept. That converts the escalation decision from an uncertain prediction into something closer to a deterministic check, which is the single biggest factor in whether a cascade works well.

The active-parameter story

Mixture-of-experts routing means a model can hold enormous total capacity while activating only a fraction of it per token. The consequence for buyers is that the parameter count in the headline and the parameter count that sets your serving cost have come apart.

Recent open-weight releases, with vendor-reported figures:

Model	Total params	Active params	Context	Reported result
Solar Open 2 (Upstage)	250B	~15B	1M	70.4 SWE-Bench Verified
Laguna-S-2.1 (poolside)	118B	~8B	1M	70.2% Terminal-Bench 2.1
MAI-Cyber-1-Flash (Microsoft)	137B	5B	256K	~96% CyberGym (with escalation)
Kimi K3 (Moonshot)	2.8T	104B	1M	67.5 DeepSWE

Every number in the right-hand column is published by the organization that made the model and has not been independently reproduced. Treat them as claims. The pattern across them is more informative than any single entry.

Two things fall out of this table.

First, the middle of the range is getting crowded and cheap. Solar Open 2 reports a SWE-Bench Verified result in the seventies while activating 15 billion parameters, with linear attention layers cutting its KV cache to roughly a quarter of a conventional model’s, and a stated minimum of four H200-class GPUs. Laguna-S-2.1 reports a comparable agentic coding score at around 8 billion active. These are models you can actually afford to run continuously.

Second, the frontier is still the frontier. Kimi K3 activates 104 billion parameters — roughly seven times Solar Open 2 — and its reported DeepSWE score of 67.5 is far above Laguna-S-2.1’s 40.4 on the same benchmark. The gap on genuinely long-horizon agentic work has not closed. That gap is precisely what the escalation path in a cascade exists to serve.

The correct reading is not “small models caught up.” It is that the distance between the cheap tier and the frontier tier now matters less than it used to, because you no longer have to pick one.

The $500 fine-tune, read carefully

Fermisense reported reinforcement fine-tuning a 9-billion-parameter open model with GRPO and having it outperform five frontier configurations on a catalog-review workflow. The reported figures: a 10.4-point lead on their benchmark, $0.50 per 1,000 listings, roughly 40× cheaper than the least expensive frontier setup and around 340× cheaper than the most expensive, and about 68× lower per-listing cost than the strongest frontier baseline. Training was roughly 1,000 GRPO steps over about three and a half days on two RTX PRO 6000 GPUs — around $500 of GPU time.

This got substantial discussion, and it deserves both the attention and the skepticism.

What is credible. The cost structure is the durable part. Two workstation GPUs and a long weekend is a genuinely different capital requirement from what task-specific model training used to demand, and GRPO is a large part of why. By estimating advantage from a group of sampled responses rather than training a separate value network, GRPO removes a model from the training loop — which cuts memory roughly in half and removes a component that is notoriously fiddly to train. That memory reduction is what puts the method on two GPUs instead of a cluster.

What is not established. The workflow was simulated, not production traffic. The evaluation has not been independently replicated. The company publishing it sells this capability. And the most suspicious detail is one they report themselves: the model crossed into the frontier performance band after roughly one day of training. Rapid crossover on a synthetic benchmark is the classic signature of a reward function that leaks task structure — the model learns the shape of the evaluation rather than the shape of the job.

What generalizes. For a narrow task with a clean, cheap reward signal, training a specialist is now within reach of a small team. That is real and it is enough.

What does not generalize. “Open models beat frontier models.” One simulated task is not evidence for that, and the same report’s own numbers on general capability do not support it.

What this means for engineering teams

The practical question is not whether cascades work in principle. It is whether your workload has the shape that makes one pay.

Measure the difficulty distribution before building anything. The entire economic case rests on an easy majority that is real rather than assumed. Sample production requests, run them through your current model, and label which ones a substantially smaller model would also have gotten right. If that fraction is not large, a cascade adds complexity and buys nothing. This measurement is a few days of work and it is the only step you genuinely cannot skip.

Pick the escalation signal deliberately. In rough order of implementation cost and increasing strength:

Signal	Cost to build	Reliability	Best when
Task-type routing on request metadata	Low	Moderate	Difficulty correlates with an observable field
Cheap classifier on difficulty labels	Medium	Moderate–good	You already collected labels in the step above
Small model’s own confidence / self-assessment	Low	Variable	Calibration has been checked on your data
Programmatic verification of the output	High	Strong	The task admits a test, compile, or check

Verification is the strongest option and it is why this pattern matured first in code and security. If your outputs can be tested, escalate on test failure and the routing problem largely dissolves.

Budget for the tail, not the average. A cascade improves average cost and average latency while making latency bimodal — handled requests are fast, escalated ones are slow. For an interactive product with a tight response budget, a better average with a worse p99 can be a regression. Decide which metric your users actually experience.

Account for the debugging cost. In a single-model system a wrong answer came from the model. In a cascade it came from the small model, the large model, or the routing decision — and distinguishing those requires logging the routing decision and its inputs from day one, not after the first incident.

Watch active parameters, not total. When comparing models for serving, the total parameter count tells you about memory footprint and the active count tells you about compute per token. Both matter, and announcements tend to foreground whichever is more impressive.

This also connects directly to a constraint we covered when memory replaced GPUs as the AI bottleneck: a cascade reduces the frontier model’s share of traffic, but the frontier model still has to be resident in memory to serve the tail. Cascades cut compute cost more cleanly than they cut memory footprint, and in a market where memory is the scarce input, that distinction matters for capacity planning.

Limitations and risks

Some honest caveats on all of the above.

Every benchmark number in this article is vendor-reported. Microsoft’s CyberGym figure, Upstage’s SWE-Bench result, poolside’s Terminal-Bench score, Moonshot’s DeepSWE number, Fermisense’s benchmark lead — all published by parties with an interest in them, none independently reproduced. The consistency of the pattern across independent organizations is better evidence than any single figure, but do not treat the specific numbers as measurements.

A single benchmark is thin evidence for a security claim. CyberGym results say something about CyberGym. Whether MAI-Cyber-1-Flash’s advantage holds on codebases outside that distribution is unknown from public information.

The 90% figure is a design target, not an audited operating statistic. Microsoft describes the model as built to handle up to 90% of tasks. What fraction it handles in practice, on real customer workloads, has not been published.

Cascades can degrade quietly. If the escalation signal drifts — because traffic changes, or because the small model is updated — the system starts silently handling requests it should have escalated. The failure is invisible in cost metrics, which will look better, and shows up only in quality metrics if you are watching them per-tier.

The cheap tier depends on the expensive tier. MAI-Cyber-1-Flash descends from a larger internal model. Small specialists are routinely distilled or fine-tuned from frontier systems. This is not an independent cheap-model industry; it is a downstream one.

Predictions

Clearly marked as predictions rather than reporting:

Cascade routing becomes a named, sold product feature at multiple vendors within a year. Microsoft demonstrated a roughly 50% cost reduction and published it. That is a competitive disclosure, and the response will be imitation.
Model cards start foregrounding active parameters and serving cost alongside benchmark scores, because for a growing share of buyers that is the binding number.
Routing and calibration become a distinct engineering specialty. The hard part of a cascade is deciding when to escalate, and that is a calibration problem that most teams are currently not staffed for.
The first well-publicized cascade failure will be a quality regression discovered late, caused by drift in an escalation signal that nobody was monitoring per-tier.
Task-specific reinforcement fine-tuning moves from research skill to ordinary engineering practice, in the same way that fine-tuning small classifiers did a decade ago, driven by the collapse in capital requirements rather than by any new algorithm.

Practical takeaways

Measure your traffic’s difficulty distribution before designing a cascade. If there is no easy majority, there is no cascade.
Compare models on active parameters when estimating serving cost. Total parameters tell you about memory, not compute per token.
Prefer verification over confidence as an escalation signal wherever the task admits a check. It converts a prediction problem into a deterministic one.
Log the routing decision and its inputs from the first deployment. You cannot debug a cascade retroactively without them.
Track quality metrics per tier, not just in aggregate. Aggregate quality can hold steady while the cheap tier quietly degrades and takes more traffic.
Check whether your latency budget survives a bimodal distribution before optimizing for average cost.
If you have a narrow, high-volume task with a programmatic reward signal, price out a specialist fine-tune. The capital requirement is now two GPUs, not a cluster.

Conclusion

The interesting thing about this week’s results is not that a small model did well on a benchmark. Small models have been doing well on benchmarks for a while, usually in ways that did not survive contact with production.

It is that Microsoft published a cost reduction as an architectural claim — we restructured what runs where, and it cost half as much — and that the architecture in question assumes the frontier model is an escalation path rather than a default. That reframing has been circulating in serving teams for a year. Having a hyperscaler state it as strategy, with a number attached, is what turns a private optimization into an industry pattern.

The frontier still matters, and the gap on genuinely hard, long-horizon work is not closing. What is changing is how much traffic has to cross that gap to get an answer. For most workloads, the honest answer turns out to be: not much of it.

References and further reading

Frequently asked questions

What is a cascade architecture in AI serving?

A cascade is a serving topology in which a small, cheap model attempts every request first and only escalates to a larger model when some confidence or difficulty signal says it should. The idea is old — classifier cascades appear in computer vision decades ago — but what changed in 2026 is that vendors began stating it as product architecture rather than treating it as an internal cost optimization. Microsoft’s MDASH harness is the clearest public example: a 5-billion-active-parameter security model attempts the work, and a frontier model is reserved for the fraction it cannot handle. The economics work because task difficulty in most real workloads has a very long tail. If the vast majority of requests are easy and a small minority are genuinely hard, paying frontier prices on every request means overpaying on nearly all of them. The engineering difficulty is almost entirely in the escalation decision.

How many parameters does a model actually need to be useful now?

Far fewer active parameters than headline totals suggest, and the gap between those two numbers is the story. Mixture-of-experts models route each token through a subset of their weights, so total capacity and per-token compute have come apart. Upstage’s Solar Open 2 has 250 billion total parameters but activates roughly 15 billion per token, with a published SWE-Bench Verified score of 70.4. Poolside’s Laguna-S-2.1 has 118 billion total and activates around 8 billion, reporting 70.2% on Terminal-Bench 2.1. Both are vendor-reported rather than independently reproduced, but the direction is consistent across independent releases: capability that required frontier-scale dense compute two years ago is reachable at single-digit-billions of active parameters for well-scoped tasks. Serving cost tracks active parameters far more closely than totals.

Can you really fine-tune a competitive model for a few hundred dollars?

For a narrow task with a clean reward signal, the evidence suggests yes, with caveats. Fermisense reported reinforcement fine-tuning a 9-billion-parameter open model using GRPO for roughly $500 of GPU time — about 1,000 steps over three and a half days on two workstation GPUs — and reported it outperformed five frontier configurations on a catalog-review workflow at dramatically lower per-unit cost. The caveats are that the workflow was simulated rather than production traffic, the result has not been independently replicated, and the publishing company has a commercial interest in it. The claim that generalizes is narrow but significant: the capital requirement for a task-specific specialist has fallen to roughly two GPUs and a long weekend. The claim that does not generalize is “open models beat frontier models.”

What is GRPO and why does it keep appearing in these results?

Group Relative Policy Optimization estimates the advantage of a response by comparing it against a group of other sampled responses to the same prompt, rather than by training a separate value network to predict expected reward. Removing the value model roughly halves training memory and eliminates a component that is difficult to train stably. That memory reduction is what brings the technique within reach of two-GPU setups instead of clusters. GRPO recurs in small-specialist results because it fits the problem shape: narrow tasks usually have a programmatic reward signal available, which is exactly where reinforcement learning works well and where supervised fine-tuning on a fixed dataset tends to plateau.

When should I not use a cascade?

When the escalation decision is as hard as the task, when the tail is not actually small, or when latency variance is unacceptable. If you cannot cheaply tell whether the small model got it right, you either run both models on everything — costing more than just using the large one — or accept silent errors on requests that should have escalated. If your workload is uniformly difficult rather than long-tailed, there is no cheap majority to capture. And because escalated requests take substantially longer, a cascade converts a predictable latency profile into a bimodal one, which matters for interactive products even when average latency improves. Cascades are also harder to debug, because a failure can originate in either model or in the routing between them.

Does this mean frontier models are becoming less important?

No — the cascade framing makes the opposite case. In a 90/10 split the frontier model handles the requests that genuinely matter and that nothing cheaper can do. System quality is bounded by it on exactly the hard cases where quality is most visible. What changes is the volume of traffic sent to it, and therefore the shape of the market: frontier capability becomes a smaller fraction of tokens served and a larger fraction of value per token. There is also a dependency worth naming — the small specialists in these results are frequently distilled or fine-tuned from larger models, and Microsoft’s security model is explicitly derived from a mid-training checkpoint of a larger internal model. The cheap tier is downstream of continued frontier investment.

How do I decide what to route where?

Measure the difficulty distribution of your actual traffic first, because the economic case depends on the easy majority being real rather than assumed. Sample production requests, run them through your current model, and label which ones a substantially smaller model could also have handled. If that fraction is not large, stop. If it is, the next decision is the escalation signal, in rough order of implementation cost: task-type routing on request metadata, a cheap classifier trained on your difficulty labels, the small model’s own calibrated confidence, and programmatic verification of the output. Verification is the strongest when available, because it converts an uncertain prediction into a deterministic check — which is precisely why the pattern matured first in code and security, where outputs can be tested.