“I think that in most timelines, humans will simply be irrelevant and extinct,” one interviewee said.
I heard that a lot. A few interviewees — mostly employed by frontier AI labs — expected the world to become unimaginably strange in a good way. One put a 30% chance on utopia, a 30% chance on dystopia, a 30% chance on extinction, and a 10% chance on something too weird to imagine. Several refused to make any prediction; several more said only that they still had hope. About half echoed one of my most blunt respondents: “I don't think humanity is going to make it.”
What is it like to live in a world you believe is about to end?
Karol Banach
Death was already inevitable
“I was born with a terminal condition,” said Matthew Gray, a board member at the existential risk community-building nonprofit Lightcone Infrastructure. "We call it aging. I've since picked up another. We call it multiple sclerosis. And AI is a third one on top. I'm not very worried about degenerating from multiple sclerosis because I'm pretty sure the robots will kill me first, just like I wasn't that worried about aging-related deterioration because multiple sclerosis will get me first."
From this perspective, AI doomers don't face a new problem; they face the oldest problem humanity has ever faced.
I pushed back. If I look at an actuarial table, I can expect another 47 years of life. I'd be pretty upset to discover I had only five.
This, my interviewees thought, was naive. Even without AI risk, I could have been hit by a car; I could have gotten cancer; I could have been nuked in a hot war between Russia and the United States. It’s not that the difference in probability doesn’t matter. It's worse to be certain that I'll die in five years than to have a 50% chance of not hitting my allotted 47. But because my death has always been an inevitability, I have been coping all along with the precarity of my existence. From this perspective, AI risk isn’t shocking and unfamiliar; it’s a significantly worse version of a problem I already know I have to deal with.
"I was never guaranteed that I was going to get a long life and a long future and a chance to meet my grandchildren,” said Gretta Duleba, an independent technical AI safety researcher and former communications manager at the Machine Intelligence Research Institute. “Those were never my right. Across human history, no one has ever been entitled to the future.”
Throughout the entire scope of human experience, many of my respondents said, apocalypse has been more the rule than the exception: the Holocaust, the Black Death, the An Lushan Rebellion, the Thirty Years' War. AI doom, as many people pointed out, is the latest and the last iteration of a societal universal. AGI might be the end of the actual entire world, but it is far from the first time people have faced the end of their own individual worlds.
And AI doom is a remarkably cozy catastrophe. If you suffered through a historical apocalypse, you'd expect to starve, be raped, watch your children die in front of you, die a slow and lingering death of smallpox or plague or wound infection. The AI apocalypse — at least for those with the slack to be worried about it — takes place in a world of wealth and relative peace and technological marvels.
“Enjoy the fact that you get to have hot showers,” said Duleba. “Enjoy the fact that you get to eat delicious food. Enjoy the fact that you get to do escape rooms, which is one of my favorite things. This is great. Have you noticed how great this is?”
“There's at least some hope that AI might become good,” said Robert Herr, a former senior political staffer who is transitioning into AI policy work, “and that is a lot more than many, many billions of people in history had.”
For some people with short AI timelines, the enormity of the AI apocalypse is its own perverse source of comfort. Once, they had to worry about many things: climate change, malaria, factory farming, democratic backsliding, the fertility crisis. Now, instead of many big problems, they have one enormous problem. Worry about AI frees them from having to worry about anything else.
“When you're diagnosed with prostate cancer,” said Adam Grey, who isn't involved in AI research but who follows AI news, "a lot of the time doctors say not to bother treating it because you'll die of something else. This is the thing that's going to kill us first — us as a civilization and also personally me. It clarifies what the most important issues are.”
Ambiguous loss
Although short AI timelines can be a source of clarity, some people also struggle with the uncertainty of humanity’s fate. Duleba, who was a therapist before she switched to working on AI, told me about the concept of “ambiguous loss,” originally developed by Pauline Boss in the 1970s.
In normal grief, your loved one is dead. While it's painful, you know that it will never change. Ambiguous grief, however, occurs when a loved one is kidnapped, or is a soldier missing in action, or has slowly worsening dementia with occasional good days farther and farther apart. Your loved one's death is never really over, so you can never really grieve. You are trapped in a cycle of mourning that never resolves.
AI doom can be a situation of ambiguous loss. You can't know for sure when it will happen or whether it will happen at all — but the more you understand what's going on, many people find, the easier it is to grieve and move forward.
“The more I know about something, the less I'm freaked out always,” said Tao Lin, a member of technical staff at a frontier AI lab (and a close personal friend). “The less I know about something, the less I will be rational about it. The rational part of your brain needs information to operate, and just having more information will make you be more premeditated and system 2 about everything."
When he felt doomy, he did AI forecasting to put concrete numbers on his uncertainty. (He believes there is about a 20% chance of human extinction, a 40% chance of ”a great outcome,” and a 40% chance that "people survive and have a great time, but stuff is broadly bad.")
Other people with short AI timelines adopted the same strategy. “AI risk was an amorphous, scary thing in my mind,” said Jack Stennett, program director at ML4Good. “So I tried to put some numbers on it. What's my p(doom)? What is my estimate of what the risk is? What are the pathways to how we survive and things go really well? What are the pathways where it goes really badly? Having a bit more precision about how you expect things to go makes you feel more in control of the situation. Fears are always worse when they're poorly defined. The scariest monster is the one you can't see.”
Most interviewees emphasized that the most important thing to understand about AI risk was how little control you had over it.
“There's not much use worrying about a thing if the outcome is determined,” said Alyssa Riceman, a software engineer. "You're just going to burn a whole bunch of emotional energy not making any changes out in the world. It's only worth worrying about things if you're in a position of control over them. So go out and check if you're in a position of control, and if you are, control them.”
But what if you have partial control over a situation?
“Then you have to game out all the branches,” Riceman said. “Say 'I can do this. If I do this, what happens then?,' until everything bottoms out at either a situation you can completely control or a situation that's out of your control.”
Duncan Sabien, who is the current communications manager for the Machine Intelligence Research Institute, agreed. “I actually have no control over whether we succeed or fail,” he said. “All I can control is my own actions. And so if I am doing the best I can with what I know and what I have available to me, then that is the best I can do. I go home feeling like I'm a good person, and I get to go to sleep at night feeling like if the AI does kill us all, I did as much as I could, realistically and sustainably, to prevent it. And everything else is out of my hands. Everything else is always out of my hands.”
Living well in the apocalypse
What, then, do people decide to do?
For many people with short AI timelines, the answer is to get to work.
“If we're going to die,” said Ben Woden, operations manager at Pivotal Research, a fellowship program for aspiring AI researchers, “I would really, really prefer to die fighting.” He then clarified that by "fighting" he meant continuing to work at his job as an operations manager for an AI safety nonprofit, not bombing a data center.
But many people can't do anything about AI risk other than go to the occasional protest. They don't have the mathematical ability to go into technical AI safety research or the persuasive ability to work in AI policy. And people who are working on AI risk might still want to do something in the hours in which they aren't addressing the problem (which, for some, is admittedly, quite rare).
Adam Grey, for instance, believes “there's a greater than 50% chance that my influence on my life or the lives of people around me has an expiration date of five to six years.” He doesn’t work in AI, though, so he can’t improve humanity’s odds directly. Instead, in the meantime, he thinks “the only action I can do to benefit myself or my immediate surroundings is to have a good time.”
People working in AI safety agreed. “It's better to have died having fun on the trampoline than to have died listening to ‘Moonlight Sonata’ on repeat while I cry,” Duncan Sabien said. “I choose between universes in which I die, and I choose the one I like most.”
“Drink the good whiskey, drink the good wine," said Dave Kasten, head of policy at Palisade Research.
Drinking the good whiskey costs money. Accordingly, “I'm not as careful with money anymore,” Duleba said. “Historically, I was very careful with money, so that means I now have money to not be careful with.”
Some people with short AI timelines made radical financial decisions. “My partner's like, 'Let's be sure to save for retirement,'" said an interviewee working in AI policy, "and I'm like, 'Retirement? That's not even in my world model! That's a type error!'" A few even plan to spend down the money they've already accumulated. Tao Lin, for instance, intends to donate his savings over the next few years.
But Duleba urges people to keep some money in reserve. We might successfully pause AI development. AI could automate most jobs, causing mass unemployment, well before it causes human extinction or utopia. The future is unpredictable, she thinks, and having some money prepares you for unexpected outcomes. Adam Grey agrees and is saving the recommended percentage. “It's not that I want to be rich in the cyberpunk future,” he said. “It's that I want to have options in any future.”
As I did my interviews, I was struck by the sheer diversity of life changes people made because of AI. They went to therapy. They quit jobs they hated to spend more time with their children. They had children because they wanted to experience their children's early years, even if they would never live to see their children as adults. They travelled. (“I studied Latin for seven years, and I've never been to Rome or Pompeii. This makes no sense.”) They threw 50-person birthday parties. They learned to camp in the snow. They starred as Harold Hill in The Music Man. They learned to grind ultramarine blue paint.
“Remember that everyone dying would mostly be bad because we couldn't do stuff that's good,” said Ben Woden. “Try not to surrender before the fight has begun by letting the fear of death keep you from having a life worth living.”
“When I’m going to a gig,” he said later, “I sometimes get the sense that it's good we're getting this before it's all over. This might be the last iteration of Damnation Festival there ever is, so it's good I was here. We've seen it out on a high. Well done, everyone.”
As Duncan Sabien put it: “There's a difference between a half-built pyramid and a pile of laundry in the corner of your room. If you're living your life well, you should always die with half-built pyramids. It's worse to die with vast piles of half-folded laundry you really feel like you should have done.”
Give yourself permission not to fold the laundry, or get it out of the way as quickly as possible. But don't allow the fear of AI doom to keep you from building pyramids.
“When I embark on a project,” Sabien added, “I say to myself, 'Oh man, if I build it, it will take 10 years. But we might all die in five years. But that's not a reason not to build it if it's beautiful.'”
