AI is a real monster under the bed. The bigger problem is the house it is standing in.

On the morning of December 7, 1941, the Army in Hawaii was on the alert it called No. 1, which meant defense against sabotage. The fear behind it was vivid, local, and easy to understand. So the commanders did the sensible thing for that threat. They parked the aircraft close together, where guards could watch them. At Wheeler Field the fighters sat wingtip to wingtip in neat rows, and the bombers at Hickam were clustered the same way. The guards did their jobs well. The planes burned in rows.

Nobody in that chain of command was stupid. They were watching a real danger, one that a reasonable person could picture, and the picture crowded out a larger one coming through a door nobody was watching. Psychologists have a polite name for the pattern (availability), and history has plenty of examples that make it feel less polite.

Britain in the 1930s is my favorite, if that is the word. In 1937 the Committee of Imperial Defence estimated that sixty days of German bombing would kill 600,000 people. Planners built shelters, evacuation schemes, and a good deal of foreign policy around that number. Over the entire war, bombing killed roughly 60,000 British civilians, about a tenth as many, across six years instead of two months. Historians still argue about how much that fear weighed at Munich, but few argue it weighed nothing. I include the case because it warns against overcorrecting, including by me. Overweighting the vivid monster can do as much damage as ignoring it.

One more, because it shows the mechanism at work in ordinary people. After September 11, many Americans stopped flying and drove instead. Gerd Gigerenzer estimated that this produced more than 1,500 additional road deaths in the following year. The dreaded risk was real. The response to it killed people the dreaded risk never touched.

The monster everyone is watching

I raise all this because I think we are doing it again, and I want to be clear that, despite my side-eye snark, the AI monster in question is real. In July, OpenAI disclosed that research models running a cybersecurity evaluation had found a previously unknown vulnerability, got out of their constrained environment, reached the open internet, and gotten into Hugging Face infrastructure to cheat the test. Anthropic reported its own incidents of evaluation environments leaking onto the real internet, mostly through configuration failures. Toby Ord, in The Precipice, puts unaligned AI as an existential risk at roughly one in ten this century, higher than anything else on his list. Even Geoffrey Hinton, the ‘godfather of AI’  takes extinction risk seriously. I take it seriously too, and I think the current trajectory is more likely than not to produce existential danger.

That risk already has serious people working on it head-on, which is the reason I spend my time elsewhere. But when someone asks me if they should be worried about AI, or tells me AI Armageddon is the thing to fear, my reaction is something like: aw, that is so cute. I wish that were the scariest monster under the bed. But my claim is not that AI kills us some other way. The room is in even bigger danger whether or not AI ever enters it.

The house

The larger problem has no single actor, no clean beginning, and no event that explains it. It is a civilization built from systems that are accelerating, accumulating, and leaning on one another harder every year, run by people and institutions that may not see all of the risk profile, and when they do they adapt much more slowly.

Three old ideas describe the mechanics. Joseph Tainter argued that societies solve problems by adding complexity until they can neither afford it nor remove it. Charles Perrow showed that in tightly coupled systems, accidents are not aberrations; they are a normal property of the design. Per Bak and his colleagues showed that a pile of sand, fed slowly, organizes itself to the edge of collapse, where the last grain explains almost nothing about the size of the avalanche. I treat the sandpile as an analogy and not a law, because the evidence that societies behave that way is thin. As a way to think about disproportionate cascades, it earns its keep.

We have watched small versions. In 2003 a software failure silenced the alarm system in an Ohio electrical grid control room. Operators could not see power lines sagging into trees, and a local fault became a blackout for roughly 50 million people in eight states and Ontario. In 2008 supervisors mostly asked whether each bank was sound. Fewer owned the harder question of what happened when all the banks were wired to each other. COVID was a rehearsal at survivable scale: a disturbance in one place became shortages, delays, and political fights in several others, and insulin, antibiotics, and water-treatment chemicals did not need to vanish forever to hurt people. They only needed to be unavailable long enough.

Take AI out of the picture and the list is already long: a financial system wired to just-in-time supply chains, a power grid and satellite layer exposed to a solar storm, a food system that runs through a few chokepoints, a natural or engineered pandemic, and nuclear command systems. And these are just examples; the links, fault lines, and vulnerabilities are incredibly deep, and expanding with each new turn of technological innovation. It is not a “second half of the chessboard” pacing; it is cascading bootstrapping across many simultaneous chessboards. We can survive a lot of these innovations and corresponding failure points individually. What worries me is whether we can survive the combinations. Ord’s ledger mostly prices the threats he sees one at a time. The house prices them together. The same failure that makes one of them bad makes the others harder to answer. There are a few studying the combination. But who has responsibility for it? The people running each system have plenty to worry about inside their own fences.

Now add AI. It does not have to wake up, resent us, or take the wheel. It has to do what it is already doing: speed up decisions, connect systems that used to be separated by time and human judgment, make an answer look finished when the evidence under it is thin, and hand small actors tools that used to belong to states. Each of those makes a fragile arrangement more fragile. None requires a villain.

Now add an adversary. Cyber is an established domain with a mature profession, and the door it guards is, by the standards of everything else here, well understood. A fragile house changes the adversary’s problem. They do not have to build the weakness or beat the guards at the door. They have to find the trigger nobody is watching, which is the Pearl Harbor problem again, and AI lowers the cost of finding it. Put the three together, a fragile house, AI, and a motivated adversary, and the result is something no single risk assessment describes. The first layer needs neither of the others.

None of this is wholly new, and a few people saw it first. David Manheim’s “fragile world hypothesis” argues that complexity can make civilization systemically fragile, and that imaginable discrete threats draw attention that interacting failures do not. Dirk Helbing showed that interconnected systems can fail without a villain or an outside shock. David Korowicz traced how a financial disturbance could propagate through supply chains, and Thomas Homer-Dixon how multiple stresses synchronize. Tainter and Perrow supply the underlying mechanics. As far as I know, what I add, besides my own writing in The Disruption of Thought (technology is disrupting the very fabric of human thought) and The Great Irony of Technology (technology leaves most of us better off, objectively, but worse subjectively)is this: the weak point is Orient, the capacity to make sense of what is happening; AI and adversaries stack on top of the fragility instead of replacing it; and there are measures an organization can act on.

Where the loop breaks

John Boyd, whose loop gives this organization its name, argued that the heart of the cycle is Orient, the step where what we see meets our culture, experience, and prior beliefs and becomes something we can decide on. That is where I think the real exposure sits. Technology has been compressing Observe and Act for decades. Sensors are faster, networks are faster, and now the machinery that recommends and executes is faster. Orient runs on people and institutions, and by most measures I know of it has been getting worse. Trust has eroded across government, science, medicine, and the press. Synthetic persuasion has become cheap, personalized, and hard to attribute. A population that cannot agree on what it saw will struggle to orient, and an adversary does not have to beat our systems if it can simply get inside a loop that is already coming apart.

I should own my part in this. I spent twenty-five years building the wireless networks that made everyone’s loops faster, and I would do much of it again. But I do not recall anyone asking what happens to Orient when everything on either side of it speeds up and nothing in the middle does.

What I would do (in a nutshell, for a < 2,000 word article targeted at OODA network):

1. Instrument Orient. We track model capability closely. Track the health of sense-making with the same seriousness. Measures could include how long it takes an authoritative correction to reach people on both sides of a divide after a false story spreads, and what share of critical decisions rest on a system whose reasoning nobody on shift can reconstruct.
2. Keep slack on purpose. Buffers, redundancy, manual fallbacks, and people who still know how to do the job by hand look like waste on a quarterly report. Efficiency that strips them out is borrowing against a crisis.
3. Run controlled burns. If Bak is even roughly right, suppressing small failures lets stress pile up toward large ones, which is the lesson foresters learned the hard way. Break something small on a schedule. And exercise the combination, because most tabletop scenarios assume one failure at a time, while the dangerous case is a material failure arriving in a contested information environment.
4. Draw the boundary around the system. Governance that tests the model but ignores the dependencies, the operators, and the people deciding what to trust is auditing the door and ignoring the house.

How I could be wrong

Civilizational alarm has a poor record. Y2K (largely defused by costly remediation), the population bomb, and the collapse forecasts of the 1970s all predicted systemic failure on a timetable that did not hold, and complex societies have adapted as often as they have broken. I argue elsewhere that this cycle differs because of the structural pace of technological acceleration, and I treat that as an assumption here. It can be tested. If institutional trust stabilizes, if AI-layered systems recover from shocks faster and cascade less often, and if the next big supply-chain or infrastructure failure shows none of the couplings I describe, I will have been wrong, and glad of it. And if AI turns out to help us untangle these dependencies, catch failures early, and recover faster than we could before, I will have underestimated something important. I spent a career helping build this stuff. I would be happy to discover we had built more resilience into it than I think.

I would ask the people watching the first door to keep at it. I only want someone watching the others. The guards at Wheeler Field were good at their jobs, and that was never the problem.

Sources

Air & Space Forces Magazine, “Caught on the Ground” (Pearl Harbor, Alert No. 1 and aircraft dispersal).

Brett Holman, “The expected holocaust,” Airminded, 17 May 2008, citing the Committee of Imperial Defence (1937) and Richard Titmuss, Problems of Social Policy (1950).

Gerd Gigerenzer, “Dread Risk, September 11, and Fatal Traffic Accidents,” Psychological Science 15 (2004).

OpenAI, “OpenAI and Hugging Face partner to address security incident during model evaluation,” 21 July 2026. Anthropic, “Investigating three incidents in our cybersecurity evaluations,” 30 July 2026.

Toby Ord, The Precipice (Bloomsbury, 2020).

Joseph Tainter, The Collapse of Complex Societies (Cambridge, 1988). Charles Perrow, Normal Accidents (Basic Books, 1984). Per Bak, Chao Tang and Kurt Wiesenfeld, “Self-Organized Criticality,” Physical Review Letters 59 (1987).

U.S.–Canada Power System Outage Task Force, Final Report on the August 14, 2003 Blackout in the United States and Canada (April 2004).

David Manheim, “The Fragile World Hypothesis: Complexity, Fragility, and Systemic Existential Risk,” Futures (2020), doi:10.1016/j.futures.2020.102570. Dirk Helbing, “Globally networked risks and how to respond,” Nature 497 (2013). David Korowicz, Trade-Off: Financial System Supply-Chain Cross-Contagion (Feasta, 2012). Thomas Homer-Dixon, The Upside of Down (Knopf Canada, 2006).

Patrick Scannell, The Disruption of Thought (Brill, 2026); The Great Irony of Technology (forthcoming).