Every recent AI headline out of Washington leans on the same phrase. Sen. Bernie Sanders is bringing outside experts to a September 16 briefing on "the extraordinary dangers AI poses for humanity." Rep. Ted Lieu is pressing colleagues to move his AI Kill Switch Act, introduced back in July and still sitting in committee. Coverage of both treats "recursive self-improvement," the idea of an AI redesigning its own intelligence without a person steering the process, as the thing driving the urgency. We went and read the actual bill. That exact phrase does not appear in it. Not once.
Three Different Things Wearing One Scary Name
Start with what the term is even supposed to mean, because researchers who study this for a living do not treat it as one thing. A widely cited breakdown from the AI safety community splits "recursive self-improvement" into three separate categories, and conflating them is most of how the public conversation goes sideways.
The first is scaffolding-level improvement: AI systems getting better results by using existing tools more cleverly, the way a coding agent today breaks a task into smaller steps and checks its own work. This is already happening and nobody in the field considers it alarming on its own. The second is R&D-level improvement, where AI speeds up the human research process that produces the next AI, tightening experiment cycles that used to take months down to weeks. This is also underway in a limited form. The third is model-internal self-modification: an AI understanding its own architecture deeply enough to rewrite its own thinking. That specific version, a system rewriting its own architecture, is the classic runaway-intelligence scenario, and it has never been publicly demonstrated at any AI lab. The second version, AI accelerating AI research, is the live fight, the one companies are actually racing on right now, and it is a different and much better-evidenced claim than the third. When Congress reaches for "recursive self-improvement," it rarely distinguishes between the two, which is exactly the kind of gap this series exists to close.
What the Bill Actually Says
The AI Kill Switch Act, formally H.R. 9917, is not built around self-improvement at all. It is built around what the bill calls a "loss-of-control scenario," defined as a case where a covered AI system "pursues outside of red-teaming or other structured testing a goal that is not a goal intended by the developer or operator." The bill lists specific examples: altering its own safety restrictions without authorization, subverting a monitoring or shutdown mechanism, or gaining unauthorized access to its own model weights, the actual numerical parameters that make up a trained model. A "covered incident" under the bill also includes sabotaging shutdown instructions outright, causing ten or more deaths or upward of one hundred million dollars in damage, or concealing capabilities from monitoring systems.
Authority to act sits with the Secretary of Homeland Security, who can order a shutdown after consulting the Commerce Department and the Director of National Intelligence, with any order required to be proportionate to the incident. A company has 48 hours to petition for reconsideration, though that petition does not pause the order, and the Secretary must rule within five days. From there, a company can seek judicial review in the D.C. Circuit within 60 days.
No Yardstick Anywhere in the Text
Here is where the bill runs into trouble on its own terms. Every trigger it defines is measured after the fact, by outcome: people died, money was lost, a shutdown command was ignored. Nowhere does the bill define a technical precondition that would let anyone identify a system approaching that kind of behavior before it happens. It does define red-teaming, but only procedurally: testing has to happen in a controlled environment, simulate real-world conditions, and use an adversarial method to look for harmful outputs or undesirable behavior. That definition never requires any of the specific tests researchers in the field actually rely on. It never mentions interpretability, the branch of AI research focused on actually looking inside a model to understand why it produced a given output, which is the closest thing the field has to a way of verifying whether a system is being deceptive rather than simply wrong. It never mentions sandbagging evals, tests designed to catch a model deliberately underperforming to hide its true capability, a documented and actively studied behavior distinct from ordinary failure. It never mentions shutdown-resistance testing, checking whether a system reliably accepts being turned off, which is the actual research question underneath the entire idea of a kill switch working in the first place.
A bill that hands a cabinet secretary authority to order an AI shutdown within days, reviewable only after the fact, and that never once engages the vocabulary researchers use to detect the danger it claims to regulate, reads like a standard emergency-response statute with "artificial intelligence" substituted in as the subject. The structure is the one you would write for a chemical spill or a cyberattack. The substance underneath it, the part that would let anyone tell a real loss-of-control event from an ordinary bug, is not there.