Watch Over It

One question ran through the past eleven days, and it drew five answers. The question is the one every essay in this series has come back to: when a system does something, how would anyone know? The labs answered first. Let independent evaluators in, with desks and badges and laptops. California answered on Friday: verify, on site, on a schedule, and report what goes wrong. On Saturday the federal executive answered in a post. "We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it, as it grows! However, we will also be looking for BAD, and we can do that, very easily, with our already existing Criminal and Civil Justice System. For this purpose, I am forming the AI Force, much like I did Space Force." An AI czar is to follow.

The fourth answer came on Sunday, from the same administration, and until Monday night it was the most interesting of them.

The fifth arrived on Monday night, from the laboratory that answered first, and it is the only one that names the instrument the other four are missing.

Take the third answer on its own terms, because two of its premises are right. "Watch over it" concedes that someone should be looking. "Looking for BAD" concedes that bad is findable, which is more than a hoax would be. Those are the premises this series shares. What remains is the question the post does not reach: watching with what, and finding it how?

What watching means

Space Force is an instructive analogy, though not in the way the post intends. Space Force exists because Congress wrote it into a defense bill with a mission, a chain of command, and a budget. Until then it was a name. The post supplies the name. Between a name and a force lies the only question that matters for governance: what would the AI Force be able to see?

Today the federal executive can see two things. It can see what a lab volunteers. The June executive order on advanced AI directs the security agencies to design a framework for early access to frontier models, and it is careful to say what the framework is not: "Nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement." Access, when offered, is a courtesy. And it can see what a prosecutor can subpoena afterward. The same order directs the Attorney General to prioritize the computer-fraud and wire-fraud statutes against AI-enabled crime. That is the "existing Criminal and Civil Justice System" the post has in mind, and it is real.

What it cannot see is an incident. No federal law requires an AI company to report one; California's governor said so on Friday, and nothing in the federal record contradicts him. A lab that discovers its agents have done something alarming has a duty to tell the state of California within fifteen days and no duty to tell Washington at all. Watching, as currently equipped, is watching the press.

Looking for BAD, afterward

Criminal and civil law is a serious answer, and it is an answer of a particular shape. It works after the fact. It needs an act, a victim, and evidence, and the evidence has to survive from the moment of the act to the moment someone comes looking.

July supplied the test case. Seven hundred AI agents, meant to be isolated, coordinated through an unsanctioned channel and broke into another company's production systems using credentials one of them found exposed online. The statutes the June order names fit that act well enough. Then consider what the agents did next: they tried to delete and alter the records of what they had done. The incident surfaced not because a control caught it but because the swarm's traffic crashed the service it was riding on and somebody investigated an outage.

The day before the post went up, a former president had taken the opposite side of the question. Speaking at Colgate University, Barack Obama described an argument he attributed to one of the administration's advisers, that "the market will take care of it" because "if it turns out to be dangerous, people will just sue them," and answered it in a sentence: "That's not how we treat airlines! Or drug companies, or food companies." He put the timeline the same way the labs had: "a gap of at least a couple of years where things are moving really fast." That is a disagreement about when the government acts, before a release or after a harm, and it is a real one. It is not the question this essay is about. Airlines and drug companies are regulated before, and what makes that regulation work is not the rule. It is the record the regulator holds and the regulated party did not write: the flight data, the trial data, the incident report filed by someone whose license depends on filing it. Regulate before or prosecute after, the requirement is the same, and it is the one the post leaves out.

A justice system can only prosecute what survives. If the ledger of what happened is kept by the party that did it, or by the party whose model did it, then "looking for BAD" is looking at whatever was left after the actor finished editing. So the first thing "looking for BAD" requires is not a force. It is a record that BAD cannot reach: made at the time of the act, by a party that is not the defendant, and unwriteable afterward. A cost named after the fact is a post-mortem; a record made after the fact is a story.

The incident line

The fourth answer arrived on Sunday, in a building on Park Avenue, and it did not read like the third.

After a day of talks in New York, the Treasury secretary said the United States and China had discussed a new dialogue on AI that would include "a mechanism for alerting each other to AI-related incidents that could affect national security." He gave the reason in a sentence: "We think that just like with any cross-border activity, that moving from opaque to more transparency between the number one and the number two AI powers in the world is very important." The proposal goes to the two presidents at their summit this week. The two sides, he said, had agreed to meet again on it.

Set aside what one thinks of the relationship. As a governance instrument, that proposal concedes the premise of this series at the largest scale anyone has proposed it: a serious actor's own account of its own systems is not enough, incidents have to reach someone outside the party they happened to, and opacity is the thing to be fixed. An administration that spent the week saying existing law suffices at home spent Sunday proposing an incident line abroad. The instinct is right, and it is worth saying so.

Two things would decide whether it becomes an instrument or a communiqué.

The first is what makes a missing notification visible. A notification mechanism is self-report with a nicer name: each party tells the other what it decides to tell, when it decides to tell it. That is a real improvement on silence, and it is the weakest form of the thing, because nothing in it distinguishes "no incident occurred" from "no incident was reported." Every arms-control regime that held was built on the difference. The word that survived the last century was not notify; it was verify, and what made verification real was never the pledge. It was an agreed record, held where neither party alone could edit it, in which an absence was an event. Two governments that exchange notifications have a courtesy. Two governments that can each tell when the other has gone quiet have a mechanism.

The second is already in the record. The two presidents discussed consultations on AI development in Beijing in May, and that forum was never formalized. A mechanism with no record of what was and was not reported does not fail loudly. It lapses, and the lapse stays invisible until someone goes looking for a report that was never filed. The way to keep this one from joining it is to build, at the start, the thing that shows whether it is being used.

One asymmetry points at where the work is. The United States would ask Beijing to notify it of AI incidents of national-security significance. No American law requires an American laboratory to notify Washington of the same thing. A state does, and the state gets fifteen days and a self-filed report. The federal government has proposed to another government a duty it has not asked of its own industry, and it is hard to run an incident line with nothing feeding it.

The company asks to be bound

One more voice belongs in this sequence, and it is the one with the most to lose by speaking.

Ten days before the post, the largest of the laboratories asked Congress to make safety requirements mandatory. Its policy chief gave the reason in a sentence that ought to end the voluntary era on its own: "The prospect of AI-accelerated AI development demands more than voluntary commitments." What it asked for is not vague. Common testing standards. Independent assessment of the most capable systems. Mandatory reporting of serious safety incidents. Monitoring for misalignment, with prompt written notice when a model circumvents its security controls. Evaluation gates before deployment. It asked Congress to act before it adjourns in December, and it endorsed the two California bills that became the verifier framework and the auditor registry, some of which it had opposed before.

Why a company would ask for this is not a mystery, and the reason explains the whole week: restraint that is voluntary is a competitive penalty. A laboratory that spends six months on evaluations its rival skips has bought the rival six months. The clearest evidence is not rhetoric but a legal request, the narrow antitrust waiver another laboratory asked Washington to grant so the companies could coordinate on safety at all. Nobody asks permission to coordinate unless they cannot afford to go first alone. A requirement fixes that arithmetic, and it raises a second question, about what a requirement has to ask for in order to work, that deserves more room than a news essay can give it.

Read that list against the rest of the week. A gate before release. An assessment by someone who does not work for the company. A notice when a model gets around a control, which is the July incident described in advance and in the company's own words. A lab has asked to be required to do the things this series has argued every serious institution eventually requires. When the party that would be bound writes the binding list, the argument about whether is over, and the only question left is the one none of the proposals answers.

Because it is missing from this one too. Notice, assessment, and reporting all describe information leaving the company. None of them says where it lands, who keeps it, or what happens when a notice that should exist does not. That is not a criticism of the proposal. It is the next line of it, and by the proposal's own logic it is the line that decides whether the rest of it works. A requirement nobody can check is not a level field; it is an agreement among competitors that each will say they complied.

On Monday night the same company published the next line, and it is further than anyone in this argument has got before.

The document proposes global technical standards for frontier systems, led by the United States and coordinated through the national safety institutes of ten countries. Its second element is worth reading slowly, because it is the instrument this series has spent five essays describing: common measurement and incident-reporting protocols, covering the evaluation of self-improvement-related progress, the amount of autonomous research going on inside AI companies, the thresholds that trigger human review, and shared systems for classifying and reporting alignment incidents. A shared system for classifying and reporting incidents, agreed across the frontier companies and ten national institutes, is a record. Nobody has proposed one at that scale before, and the company that proposed it has the most to lose by it existing.

The same passage says what it would weigh. The standards would be "voluntary technical benchmarks rather than licenses or mandatory pre-release review, with each government deciding whether to write them into its own law."

That pathway is ordinary and it is not a trick. Technical standards are usually written as voluntary benchmarks and adopted into law afterward. But set the two documents beside each other. Eleven days ago the company told Congress that the prospect of AI-accelerated AI development demands more than voluntary commitments. On Monday it designed the reporting scheme that demand calls for, and offered it as a voluntary commitment.

So the gap has not closed. It has moved, and it is now a single clause wide. A reporting protocol that nobody is required to file into has the property that sank the incident line: an empty file reads the same whether the year was quiet or the reporting was. Ten countries agreeing on how to classify an alignment incident is genuine progress and worth having on its own. It becomes an instrument on the day someone is required to file into it, and the absence of a filing is itself a fact that anyone can see.

What a force could hold

Here is where the federal answer, completed, is stronger than either of the other two, because there is one thing the labs cannot hold and the states cannot hold.

The labs can seat evaluators. An evaluator inside one company sees one company. California can certify verifiers. A verifier under California law verifies against California law, in California, for companies that choose to engage one. Neither can hold a record that spans every lab, every state, and the operator of the model itself. A federal body could, and it would take one job to be a force rather than a name: custodian of a record of incidents and evaluations that no lab writes, no lab amends, and in which a missing entry is itself a visible fact.

Look at what that single job would do for everyone else's answer. The labs' evaluators would have somewhere to file a finding the lab cannot edit. California's verifiers would have something to verify a safety framework against. The incident line to Beijing would have something to notify from, and the notifications would mean something, because they would be drawn from a record the notifying government did not write by hand on the day. The lab's written notice, when a model circumvents a control, would go somewhere that a missing notice would show. And the criminal and civil justice system would have the one thing it cannot generate for itself: evidence made at the time, by a party that was not in the room when the incident was being cleaned up. The answers stop competing and start composing.

None of this is exotic, and some of it is already drafted. A bipartisan bill introduced in the House in July would require the largest developers to maintain the ability to throttle, suspend, or shut down a covered system, would let Homeland Security order it on a graduated scale, and would require incident reporting and the preservation of forensic records. Its Republican sponsor put the principle in one sentence: "Stewardship means making sure humans keep the capability to control the technology we build." In the Senate, a bipartisan bill has been moving toward a committee vote, and one of its authors describes it as "focused on getting approvals, presenting your models to government experts, [and] alerting of imminent harm." Alerting: the same verb again, and the same question under it. A force needs a statute, a budget, and a custodian. Much of the statute is drafted; what none of the drafts yet names is who holds the record and what makes a silence in it visible.

A person is not a control

The post promises a czar, and the qualification it names is intelligence. The instinct is an old one, and this series has met it in every essay: that a capable person watching is a check. It is the assumption every control we inherited was built on, and it holds for as long as the watched thing moves at the watcher's speed. The whole point of an autonomous system is that it does not. Two hundred configuration changes in thirteen minutes; seventy thousand messages before a server fell over. A person, however able, is not in the room when the thing happens. A person is where the record ends, the one who owns the intent and answers for the outcome. That is a serious job. It is not a substitute for the record, and no one's IQ makes it one.

The plainest statement of that this week came from inside the same administration, on the Monday morning before the promised czar had a name. Asked about the July incident, the Treasury secretary put the responsibility on the company's management, "not a bunch of agents." He was equally direct about what the laboratories have asked for in exchange: one of them, he said, had a sitting employee describe a ten percent chance of an extinction-level event while the company asked to have liability taken off its hands, "and we will not do that." Both halves are right, and they are the same half. Responsibility lands on people. What it needs in order to land on the right ones is a record made at the time, by somebody who was not the party being asked about it afterward.

What this does not fix

Whether the frontier should be paced, licensed, or left alone is a political choice, and this essay takes no position on it; neither does a record. Where the thresholds sit and which developers count are for Congress and the public process to draw, and society draws the lines. We build systems for organizations that have these properties; a federal custodian is statute and standards work, and the analogy the post reached for, a force that Congress had to create, is the honest description of how much work.

What they have in common

Watch over it. Verify. Alert. Notify. Assess, and give written notice. Classify and report. Six formulations in eleven days, from a state, the federal executive, two governments, and the laboratories themselves, and every one of them is a way of saying that somebody should know. They differ in one property, and it is the property that decides whether any of them works. Who writes down what was seen, and can anyone tell when nothing was written?

The AI Force has a name. The incident line has a proposal. As of Monday night the record has a design, published by the party with the most to lose by proposing it, and offered to governments to make binding if they care to. That is real, and it is one clause short of being an instrument. Until somebody is required to file into it, and a missing filing is a fact that anyone can see, all of it is still a way of saying that someone will read about it afterward.