Atomic weapons entered the world in the 1940s, alongside the first jet engines, microwave ovens, radars, and electronic computers. Virtually every other invention of that era has spread throughout the world. Nuclear weapons are the exception. While two dozen countries retain some nuclear latency as a hedge, today only nine countries have the bomb.

This outcome was hardly inevitable. Knowledge of how to build nuclear weapons quickly escaped the confines of secrecy, and over time it became clear that even countries of modest means — North Korea, Pakistan, and South Africa — could develop the bomb with sufficient dedication. International rules and pressure could raise obstacles, but countries willing to pay the cost — to “eat grass or leaves” as former prime minister of Pakistan Zulfikar Ali Bhutto once put it — would, with time and luck, succeed.

Why, then, did so few countries do so? 

Historians, social scientists, and officials have spent careers trying to understand the spread of nuclear weapons. But, at the risk of oversimplifying, we only have nine nuclear-armed states because the vast majority, after weighing costs and benefits, decided not to develop the bomb. Almost all stuck with that choice.

Despite its flaws and vulnerabilities, the nuclear nonproliferation regime may provide useful guidance as new technologies like synthetic biology and artificial intelligence come into their own. The conventional wisdom is that powerful technologies inevitably diffuse widely. The fizzling out of nuclear proliferation provides a counterexample.

After World War II, the U.S. debated how to manage its nuclear monopoly, ultimately floating a proposal for international oversight of atomic energy that had little chance amidst Cold War tensions. ¹ A U.S. intelligence report circulated in 1949 predicted that the Soviets would most likely develop a weapon by mid-1953 and that the “earliest possible date” was 1950. Just months later the Soviet Union tested its first weapon. ²

Stalin’s sprint to the bomb showed that the technological obstacles to building nuclear weapons were weaker than once thought. In a Brookings Paper entitled Predicting Proliferation, scholar Moeed Yusuf surveyed the forecasting literature from 1949 to 1964 and found that “an overwhelming majority of classified and academic studies suggested that horizontal proliferation was inevitable.”

One reason for pessimism was an assumption that the barriers were technological, not political. Denis Healey, who later became the U.K.’s defense minister, wrote in 1960 that “so far no country has resisted the temptation to make its own atomic weapons once it has acquired the physical ability to do so.” ³  

At that time, only the U.S., the USSR, and the U.K. had nuclear weapons, but the “physical ability” to develop the bomb was spreading rapidly. Many countries were planning or building enrichment and reprocessing facilities capable of generating bomb fuel. These capabilities were accelerated by widespread nuclear trade, especially via the Atoms for Peace program, through which the U.S. shared civilian nuclear facilities as an extension of its soft power. ⁴ Starting in the 1950s, a dozen countries seriously considered or pursued nuclear weapons. Early movers included Sweden, Canada, and Australia, followed by many industrialized countries in Europe and Asia. 

In a March 1963 press conference, then U.S. President John F. Kennedy warned that 15 to 25 states might obtain military nuclear capabilities by the 1970s. Author and former U.S. security official Peter Lavoy, in a review of declassified documents, notes that Kennedy “based this pessimistic forecast on a secret study that Secretary of Defense Robert McNamara had given the president one month earlier. In this document, McNamara expected that by 1973 eight new states might acquire nuclear weapons — China, Sweden, India, Australia, Japan, South Africa, Germany, Israel — and that, shortly thereafter, many more countries could go nuclear as the cost of acquiring nuclear weapons ‘may come down by a factor of two to five times.’” ⁵

Seeing a wave of proliferation approaching, the U.S. and the Soviet Union joined in a tacit partnership to limit nuclear spread. Negotiations started in earnest at the United Nations Conference on Disarmament in Geneva in 1965. 

The resulting 1968 Nuclear Non-Proliferation Treaty (NPT) is often described as having three pillars: nonproliferation, disarmament, and peaceful uses. The central focus was nonproliferation: Signatories that had already tested nuclear weapons agreed not to share nuclear-weapons technology and other signatories agreed not to acquire weapons; safeguards would be provided by the International Atomic Energy Agency (IAEA). On disarmament, the nuclear states agreed to “pursue negotiations in good faith” to end the nuclear-arms race and pursue disarmament under “strict and effective international control,” a mandate with no timeline. Finally, the treaty called for technical cooperation toward peaceful uses of civilian nuclear technology. ⁶

To say the Nuclear Non-Proliferation Treaty had limitations would be an understatement. It lacked universality and effective verification. It had a duration of 25 years and would require extension every subsequent five years. Two nuclear-armed states, France and China, declined to join, as did a number of other states, including some with active weapons programs: Argentina, Brazil, India, Israel, Pakistan, and South Africa.

So while it is tempting to see the NPT as a turning point in the proliferation story, the treaty in its early years was akin to a white picket fence — more signal than barrier. States continued to hedge, adding to their nuclear capabilities without crossing the line into weaponization.

Intelligence analysts and outside experts remained pessimistic that nuclear spread could be contained. A threshold was crossed in 1967 when Israel secretly built a bomb, shrouded in ambiguity. Then in 1974 India detonated what it claimed was a “peaceful nuclear explosive” (code-named Operation Smiling Buddha). The dam seemed poised to break, and the U.S. Central Intelligence Agency predicted that 10 other nations had the potential and incentives to go nuclear. ⁷

But that never happened. After 1974, only North Korea, Pakistan, and South Africa acquired nuclear weapons. South Africa dismantled its stockpile of six warheads in 1989. Belarus, Kazakhstan, and Ukraine traded away the Soviet weapons stranded on their soil in return for economic and security assistance. Illicit programs in Iraq, Syria, and Libya notwithstanding, the system has held up surprisingly well for the past 50 years.

India’s test was something of a pivotal moment in galvanizing international action. It was a demonstration that countries not aligned with Washington or Moscow could get nuclear weapons, and it led to a loosely coordinated response.

First, the major powers leaned on their allies. Over the coming years, the U.S. reiterated security guarantees for allies in Europe and Asia who would forego nuclear weapons, and the Soviet Union suppressed nuclear aspirations among its Warsaw Pact allies. These arrangements were transactional: To get protection from NATO or the Soviet Union you needed to forego nuclear ambitions.

Second, nuclear exporters reinforced the thin filament of the NPT into a thicker web of regulations and controls. The first strand was the 1974 “trigger list,” specifying which nuclear items required IAEA safeguards to export. The Nuclear Suppliers Group, which began in 1974 as the London Club, formed to set voluntary guidelines on sensitive exports. NPT membership expanded from 46 to 91 countries by the second conference in 1975. Signatories to these agreements adopted international guidelines into domestic law. The IAEA increased its professional staff and competency with the support of its member states.

All these steps represented a major evolution in the nonproliferation regime. The restrictions written out in the NPT were now backed by legal, logistical, and political barriers that increased the expense, time, and risk to build a bomb. These measures held up surprisingly well, even in the face of the sophisticated illicit network established by A.Q. Khan, often called the father of Pakistan’s atomic-weapons program.

These mechanisms worked in part because they were backed by the threat of sanctions and military force. Israel conducted air strikes against nuclear reactors in Iraq (1981) and Syria (2007). While the U.S. has cautioned cheaters that “all options are on the table,” its favored counterproliferation tool has been multilateral sanctions, which the UN authorized against Iraq, Iran, and North Korea. Sanctions have a mixed record with hard cases but the potential of sanctions has served as a deterrent to other states. ⁸

These three factors — security guarantees, international controls, and counterproliferation pressure — are mutually reinforcing. Safeguards and the threat of sanctions or military force raised the costs of acquiring nuclear weapons while military pacts decreased their benefits by allowing states to achieve security through other means.

But a fourth factor may be more significant still: the establishment of a global nonproliferation norm. The NPT, with 191 signatories, has close to universal participation, and for the vast majority of its signatories nuclear weapons have little value as military assets or political tools. 

The norm against acquiring nuclear weapons is closely tied to the taboo against using them, which began to emerge as horrific details from Hiroshima and Nagasaki became public. Leaders, presented with what nuclear war would really entail, often recoiled and sought other paths to security.

The emergence of nuclear-weapon-free zones (NWFZ) helped codify and sustain these norms. NWFZs allowed states to commit to not manufacture, acquire, test, or possess nuclear weapons. The first of these, the Treaty of Tlatelolco (1967), was eventually signed by all 33 countries in Latin America and the Caribbean. ⁹ Others followed suit in the South Pacific, Southeast Asia, parts of Africa, and Central Asia. By ratifying the NPT and joining a nuclear-weapon-free zone, countries could credibly signal to their neighbors that they weren’t seeking nuclear weapons, solving a collective action problem that might have otherwise led to costly and counterproductive arms races.

Norms remain important because nonproliferation relies so heavily on state intent. Under a basic IAEA safeguards agreement, countries declare which materials and facilities to submit to inspection. The IAEA cannot impose conditions on states other than those willingly accepted. The flaws of this approach became apparent in the 1990s when the IAEA discovered that Romania and North Korea had clandestinely extracted plutonium and that Iraq had a covert nuclear-weapons program (which was destroyed by special UN inspection teams after the 1991 Gulf War). In response, the IAEA established an Additional Protocol that allows inspectors more access to data and timely inspections — but this stricter set of rules is only in force with the 140 states that have voluntarily accepted it.

A careful cheater is likely to succeed given enough patience. The NPT gives wide leeway when it comes to “peaceful uses” of nuclear power, which may include national enrichment and reprocessing programs. This means states can remain compliant with their NPT and IAEA obligations while coming a screw’s turn from assembling a weapon. Iran was caught pursuing a secret nuclear-weapons program, but had it remained patient and not prematurely pursued weaponization it might have already succeeded.

The international community was able to limit the “horizontal” spread of nuclear weapons by increasing the costs and diminishing the benefits of nuclear weapons with security guarantees, international treaties, export controls, sanctions, military action, and norms. In contrast, efforts to limit “vertical” proliferation within nuclear-armed states was less successful. Strategic stability and arms control slowed the pace of nuclear expansion, but when the Cold War ended, the U.S. and the Soviet Union were still targeting each other with a staggering 57,000 nuclear weapons.

Nuclear weapons are often invoked in conversations about transformative AI. In May, the founders of Open AI published a memo arguing that “we are likely to eventually need something like an IAEA for superintelligence efforts.” Separately, the Center for AI Safety issued a statement, endorsed by prominent AI experts, that said: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

What would it mean to treat AI as a societal-scale risk? It’s not clear. AI is in a similar place to nuclear science in the 1930s: The scientists at the frontiers of nuclear technology could see the potential for harm, but nobody knew what form it might take. No one had seen a mushroom cloud over Hiroshima or a meltdown in Chernobyl; to suggest such a thing was possible would have put you in the company of science fiction writers and cranks.

Analogies between nuclear technology and AI can quickly break down. Let’s take the concept of “an IAEA for superintelligence efforts.” The IAEA exists to help states safely and securely operate nuclear technology and to monitor those activities to ensure they remain peaceful. The IAEA can perform this task because there’s a clear understanding of what constitutes safe civilian reactor operation and what constitutes failure (accidents and meltdowns). There is also an understanding of what nuclear activities are most susceptible to proliferation, how to monitor them, and what’s at stake. In contrast, there is no consensus yet around which AI activities are unsafe, or whether we should monitor them. Separating military from peaceful uses is not the primary challenge, since AI safety experts are equally concerned about powerful yet misaligned civilian systems.

Despite these differences, I see three takeaways from those seeking to manage AI risk:

First, we should not despair if initial rules and regulations appear too weak. Assuming sufficient warning signs, regulations can become stronger with time. It took three decades — and wake-up calls such as India’s nuclear test (1974) and the discovery of a secret weapons program in Iraq (1991) — to galvanize action toward a stronger and more universal regime.

Second, the key to effective regulation is understanding which capabilities are harmful and what the key choke points are. For nuclear weapons the limiting factor is highly enriched uranium and plutonium. These materials are produced in fuel-making and reprocessing plants — facilities that require special scrutiny. 

With AI, the limiting factor is most likely computational resources. Current models take hundreds of millions of dollars to train, and the requisite server clusters can be located and observed. It is hard to distinguish whether these computational resources are being used for good or ill, or ways to identify which applications pose the greatest risks. 

Finally, norms matter. The nonproliferation regime has been successful because leaders chose not to pursue capabilities within their reach. Could norms of precaution around powerful and recursively self-improving AI systems emerge?

The nuclear taboo came about after harms were evident: terrible human suffering in Hiroshima and Nagasaki. AI, unlike nuclear weapons, has widespread civilian applications and promises enormous benefits to society. It would be undesirable to block all further AI development, even if doing so were possible. But we need norms against deploying technologies with clear pathways to misuse and those that could escape human control.

Recent opinion polls and U.S. Senate hearings suggest that citizens and policymakers are open to restrictions that embed caution. Leaders of key AI-development labs, such as OpenAI CEO Sam Altman, DeepMind CEO Demis Hassabis, and Anthropic CEO Dario Amodei, have expressed concerns over safety. So has Microsoft founder Bill Gates. These nascent norms could contribute to corporate self-restraint

While we are only eight decades into the nuclear age — and entering a new and risky moment — the story so far offers hope. In 2019, Sam Altman paraphrased Robert Oppenheimer, the father of the atomic bomb, saying “technology happens because it is possible.” We see a possible future of AI competition — between commercial labs and between the U.S. and China — that could unleash dangerous capabilities and a race to the bottom. But the limited spread of nuclear weapons offers a lesson: The possible is not inevitable.