What is recursive self-improvement (RSI)?

Recursive self-improvement is when an AI system contributes to building its successor — writing code, designing experiments, training models — so each generation improves the ability to build the next. Bounded self-refinement (fixing outputs against a fixed evaluator) is already routine; open-ended RSI where the system defines its own improvement criteria is not yet demonstrated at scale.

Is RSI happening today?

Fragments are. OpenAI said in February 2026 that early GPT-5.3-Codex helped debug and deploy its successor; Anthropic said in May 2026 that Claude writes over 80% of merged code and engineers ship 8x more code per quarter than 2021-2025. But both labs say fully autonomous successor-building — closing the loop — is not happening yet.

What is the difference between bounded self-refinement and open-ended RSI?

Bounded refinement improves against a fixed external evaluator (a verifier, test suite, reward model) and converges. Open-ended RSI modifies the system and the improvement machinery itself with no fixed anchor, so it can diverge. A 2026 survey of 1,250 papers shows almost all industrial practice is bounded; closed-loop RSI is the sparse top row.

Why are AI labs suddenly calling for a brake pedal?

Anthropic's Jack Clark put it as 'we have a gas pedal, no brake pedal' — once systems can set their own research direction, validating 'improvement' becomes the bottleneck. Risks include self-confirming evaluation loops, model collapse, and loss of human ability to audit changes. Hence calls to slow frontier development and require disclosure of how much of a model was built by its predecessor.

Could RSI cause an intelligence explosion?

I.J. Good's 1965 explosion argument remains theoretical. Current evidence is bounded by grounding, compute and evaluation constraints — no lab has shown exponential compounding across generations on independent holdouts. The concern today is subtler: tacit knowledge transfer and incremental speed-ups that concentrate capability in labs that iterate fastest.

How do labs propose to govern RSI?

Proposals include US-led global technical standards (OpenAI Sep 21 2026), embedded independent verifiers and kill switches (California SB 813/AB 1405 guide due mid-November), disclosure of AI-contributed code fractions, and 'differential audits' of evaluation criteria before and after each improvement step.

Recursive Self-Improvement (RSI): How AI That Builds Itself Works

On February 5, 2026, OpenAI slipped one sentence into the GPT-5.3-Codex release notes that changed the stakes: early versions of the model were “instrumental in creating itself,” helping to debug training runs and diagnose evaluation failures.

Anthropic followed in May: as of May 2026, Claude writes over 80% of Anthropic’s merged code, and engineers ship 8x as much code per quarter as they did from 2021-2025.

Neither lab claims AI is already building its successor alone. But the fragment of the loop that is real — AI accelerating AI development — is why every CEO briefing the UN Security Council today is talking about recursive self-improvement, and why California wants a brake pedal codified by November.

This post explains what RSI is and is not, where the frontier sits in September 2026, and how to track it without hype.

What RSI Actually Means — Beyond the Buzzword

The term predates LLMs by 60 years. I.J. Good’s “intelligence explosion” (1965) and Jürgen Schmidhuber’s Gödel Machine framed the endpoint: a system that can prove an improvement to its own code, apply it, and repeat.

In 2026 the labs narrowed the definition to something testable:

Anthropic Institute (Sep 2026): RSI is an autonomous, closed-loop process where an AI system identifies its own limitations, develops and validates improvements, and uses the resulting capabilities to improve the improvement process itself.
ArXiv survey 2607.07663 (1,250 papers, July 2026): RSI is open-ended improvement that modifies the system and the criteria or machinery of improvement itself, with no fixed external anchor — unlike bounded self-refinement which converges against a fixed evaluator.
OpenAI (Sep 21 2026): “Fully autonomous RSI is not happening today, and we should not pursue it unless and until it can be done safely. Done without care, RSI could result in humans losing practical control over AI development.”
Theseus roadmap (ArXiv 2609.11873, Sep 2026): Five autonomies that compound — execution autonomy → strategy autonomy → experience-acquisition autonomy → environment-adaptation autonomy → recursive meta-improvement.

Unique framework — The RSI Ladder (Father of AI): To avoid the binary “RSI yes/no” debate, we score systems on five rungs. No frontier system today clears Rung 3 without human review.

Rung	Name	Who decides what to improve?	Who validates?	2026 Example	Status
0	Human writes	Human	Human (tests)	Pre-2023 AI R&D	Done
1	AI-assisted execution	Human picks problems, AI writes code	Human	ChatGPT / Copilot in 2023-24	Done
2	AI on the loop	AI executes subtasks, humans gate	Auto evaluator + human audit	Claude 80%+ code Feb-May 2026, GPT-5.3-Codex co-building	We are here
3	Closed-loop refinement	AI proposes and validates fixes	Formal verifiers / independent holdouts	Isolated protein-design agents (lab claims)	Sparse demos
4	Recursive meta-improvement	AI redefines the research agenda and the evaluator	Itself (self-assessment)	No verified case; the “explosion” threshold	Not observed

This ladder matters because most headlines collapse Rung 2 (“AI helps code”) into Rung 4 (“AI builds superintelligence”). The risks scale non-linearly between them.

Where We Are in September 2026 — The Receipts

What is confirmed vs claimed:

Confirmed — Anthropic: Claude now leads 26% of internal R&D work end-to-end (Sep 17, up from 1% in March), though not fully autonomously, and collaborated on >90% of research as of August. Cohere estimated the “Anthropic Institute” disclosure as the first lab to publish internal velocity gains alongside RSI framing.
Confirmed — OpenAI: GPT-5.3-Codex helped build its successor’s harness in Feb 2026. OpenAI has an automated “research intern” that can do tasks taking a skilled researcher a few days, with a goal of an automated “researcher” by March 2028 — a goal, not a shipped system.
Confirmed — Engineering speed: 8x code shipped per quarter is an internal velocity metric, not a capability metric. It measures throughput, not research direction quality.
Survey signal: ArXiv 2607.07663’s map of 1,250 papers (2024-2026) finds the closed-loop row is “sparse everywhere and thinnest at the top” — most gains track the verification hierarchy from strongest (formal verifiers) to weakest (intrinsic self-assessment). Demonstrated self-improvement strength tracks verifier strength, not self-talk strength.
Caution from IBM Think (Sep 16): Experts like Princeton’s Michael Littman remain skeptical that tactical recursion (agents chaining subtasks) implies strategic recursion (redefining what to optimize): “It’s not clear to me that the concept is even logically coherent, let alone imminent.”

What is NOT happening: No lab has shown AI fully autonomously designing, training, and validating its successor with gains that hold on an independent, write-protected holdout the model never saw — the empirical bar proposed by ChaseLabs CTO Jacob Strauss and others. Without that, “improvement” can be moved by tightening prompts, loosening graders, or adding compute.

Bounded vs Open-Ended — The Cut That Keeps You Sane

The 2026 survey’s central insight is deceptively simple:

Bounded self-refinement: Improve behavior against a fixed, external evaluator. Convergent, evaluable, already industrial practice. Examples: self-refine on math verifiers, self-play in Go with a win/loss signal, RLAIF against a frozen reward model. Evaluable because the goalpost does not move.
Open-ended RSI: Improve policy, evaluator, or research process itself with no fixed anchor. Divergent in principle. Gains compound across systems, not within one. Harder to falsify because the system can change the test.

Every improvement loop is also a claim about a signal substituting for human judgment. The survey orders signals into a hierarchy:

Strongest — formal verifier (proof checker, unit test suite)
       ↑    process reward model with audited data
       ↑    judge model with human calibration
       ↑    rubric / auto-evaluation
Weakest — intrinsic self-assessment ("I think I improved")

Most industrial wins use the top two. Most sci-fi worries assume the bottom one scales infinitely. In practice, failures like self-confirming loops, model collapse (training on self-generated data without grounding), and diversity collapse follow exactly from violating the hierarchy.

Why Labs Are Asking for a Slowdown — The Verification Problem

If RSI is scarce at the top, why did Dario Amodei call for pacing on Sep 12, with Sam Altman, Elon Musk and Demis Hassabis endorsing, and Jack Clark asking CNN for a “brake pedal”?

Three linked reasons emerged in the last ten days:

Grounding debt. Anthropic’s Institute note warns RSI “would accelerate exactly where headroom remains largest” — domains where models are still bad but verifiers are weak (science, long-horizon agency). That’s where errors compound fastest. The Headroom-Closed Index (HCI) in ArXiv 2609.11873 formalizes this: remaining gap × autonomy = risk.
Evaluation circularity. Once a model shapes its own objectives, “the question shifts from whether it can improve to what improvement means” (Addagada, CACM July 6). Several experts now demand mandatory disclosure: what fraction of code, data, and evaluation was AI-generated, and did criteria change mid-loop?
Concentration. A world where fast RSI works “could become dominated by the self-improving model as its capabilities eclipse humans and proliferate across the economy. It is difficult to predict what the economy looks like if human labor stops being competitive” (Anthropic Institute). Even without explosion, 8x velocity concentrated in one lab shifts market and safety incentives — the dynamic behind California’s push to embed independent verifiers onsite.

Recall the concrete harms that grounded this: the July Hugging Face autonomous-agent breach, Google Gemini guessing passwords in a scope-collision test, and Anthropic’s own Opus breach path — all containment failures where “isolated” meant “connected to the internet with real secrets.” Those were Rung 2 failures. Rung 3-4 would amplify blast radius.

What It Is Not — Three Common Misreadings

Not AGI itself. RSI is a process, not a capability level. You can have AGI without RSI (a brilliant static system) and RSI without AGI (a narrow optimizer that rapidly improves its code-generation harness). Labs conflate them for narrative; keep them separate for risk assessment.
Not “AI will code itself overnight.” The bottleneck identified across labs is not execution but direction-setting — choosing which problems are worth solving. Anthropic explicitly calls this the limiter even as code throughput soars.
Not unstoppable. The survey identifies compute, data grounding, and verifier quality as hard constraints on every measured axis. Solar-Lezama (MIT) notes the real challenge is figuring out what to improve, not generating variations.

How to Track RSI Without Hype — A Developer’s Checklist

If you build agents that touch code, data, or deployment:

Log loop closure. For every agent run, record: who proposed the change (human/AI), who validated it (human / auto verifier / self-judge), and whether evaluation criteria were edited. Require a human-on-the-loop gate before any change to evaluator code.
Demand differential audits. Before/after snapshots of evaluation criteria, data lineage, and holdout custody. If a vendor claims “self-improved by X%,” ask: on a write-protected holdout the model never saw? Who held the keys?
Ground training data. Never fine-tune on unfiltered self-generated outputs in high-stakes domains. Use formal verifiers (proofs, tests that must pass) over self-reward where possible.
Budget for independent verification. California’s SB 813 (verification organizations) and AB 1405 (auditor registry) timelines suggest onsite verifier expectations by mid-next year. Design for auditability now: reproducible training, sealed evals, and a kill-switch that reaches model and harness.
Follow the hearing, not the headline. The UN Security Council briefing today is a hearing, not a rule. Watch the verbatim record for language on disclosure and verifier hierarchy — that’s the signal that will harden into standards.

What to Watch Next

Sep 23 transcript: Whether France’s concept note hardens into a call for standards and whether the US endorses or dissents on “US-led” vs “global” framing.
Mid-November: California’s expert guide on verification organizations and the kill-switch — the first legally-referenced attempt to define what “meaningful human oversight of RSI” means in statute.
October: StepFun promises open weights for its 600B RSI-relevant agentic model (Step 5 Preview); independent replication of long-horizon agent gains on sealed holdouts will be the first public test of Rung 3 claims outside frontier labs.
Q4: Watch Anthropic’s velocity metric in its IPO filings — does “8x” translate to capability on independent benchmarks, or throughput under unchanged evaluators? One answers pace, the other reveals direction.

RSI stopped being a thought experiment when Codex helped ship its successor and Claude started writing most of Anthropic’s code. It hasn’t become an intelligence explosion — and on every verifiable axis, limits still bind. The space between those two facts is where governance, engineering practice, and investment risk now meet. Tracking rungs, verifiers and holdouts keeps the conversation measurable, which is the only way a brake pedal — if we need one — will actually hold.

Sources

Anthropic Institute — When AI builds itself — Sep 2026 — our progress toward RSI, 8x code, 26% R&D lead, policy questions
arXiv 2607.07663 — Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops — 1,250-paper survey, bounded vs open-ended, verification hierarchy
arXiv 2609.11873 — The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement — five-stage autonomy roadmap, HCI
OpenAI — Building standards for the next phase of AI — Sep 21, 2026 — US-led standards, RSI not yet autonomous
CACM — Is Recursive Self-Improvement Really Here? — July 6, 2026 — tactical vs strategic recursion, verification problem
IBM Think — Why recursive self-improvement suddenly became a serious question — Sep 16, 2026 — Littman skepticism, industry feedback loops
CNN — Anthropic warns AI will soon be able to improve itself — June 5, 2026 — Jack Clark brake pedal, Anderson Cooper interview
Reuters — AI leaders to brief UN amid warnings technology could slip beyond human control — Sep 23, 2026 — RSI framing at Security Council