Even before the latest batch of alarming AI news, a lot of Americans were souring on AI. That may be for big, existential reasons — concerns about widening inequality, mass job loss, and the still-unclear environmental impact of data centers — or for very specific, personal, everyday ones, like being forced to use a worse version of Google search. 

None of this is to say that AI cannot be a useful tool for some people some of the time. It’s just that the untested technology was thrust upon millions of people who didn’t necessarily ask for it and really seem to not like it. More than twice as many Americans distrust AI as trust it, and a growing number of people think it will have a negative impact on productivity and economic growth, according to a recent survey by Gallup and the Special Competitive Studies Project. While about half of Americans now report using chatbots, a majority also think that AI will put their personal information at risk and nearly half think it will have a negative impact on society, according to a 2026 Pew study.

“It’s being forced on users, there’s no clear way to turn it off, and it feels like this unwelcome addition to their lives,” Rory Mir of the Electronic Frontier Foundation (EFF), a nonprofit that defends digital privacy and free speech, told me. “It’s this kind of continued pattern of disrespect to users that I think is part of the pushback.”

If you want less AI in your life, opting out is not always easy. But there are things you can do to minimize your exposure. 

There are ways to turn off AI features, although the process can be tedious. There are also steps you can take to prevent your data from being used to train AI models, though it’s difficult to keep all of your online data away from web scrapers. 

I talked to five experts — including a computer scientist and a researcher studying AI resistance — who offered distinctive perspectives on what AI is doing to us, and who shared some simple and concrete steps you can take to push back against AI’s creep into our daily lives.

The AI that’s suddenly everywhere

In the early days, which were just a few years ago, you really had to know what you were doing if you wanted to test out AI. Now, it’s impossible to avoid. 

“More and more AI use is through its integration into digital products like search, email, camera apps, and social media,” Arvind Narayanan, a computer science professor at Princeton University, told me. “It will become increasingly hard to strip these out unless one is willing to forgo digital products altogether.”

What you’re specifically encountering is generative AI, and one of the most common types is a subset called large language models (LLMs), which ingest huge amounts of data and learn statistical patterns to predict and generate text. ChatGPT is powered by an LLM. The AI Overviews that sit on top of your Google search results come from an LLM. The grammar checker in your email app is likely powered by an LLM. Other subsets of generative AI include models that can produce images, video, audio, and software code. 

The data that makes many of these tools work has been scraped from the open web, pirated books, and rare print books that are subsequently being destroyed. Everything from Wikipedia articles to YouTube videos has become training data for LLMs, and in order for the models themselves to get better, AI companies need more fresh data, which is in increasingly short supply. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent. Future Perfect is also funded in part by the BEMC Foundation, whose major funder was also an early investor in Anthropic; they don’t have any editorial input into our content.)

As a result, some of these companies are venturing into their users’ profiles to find more data. Meta, for example, sparked global outrage in 2024 when it informed some Facebook and Instagram users that it was using their public photos and text to train its AI models. Meta still does this. That your family photos or social media updates are being used to train AI to look and sound more human is, at the very least, very dehumanizing.

Generative AI, of course, is not the only type of AI that’s invisibly moving levers in your digital life. Recommendation algorithms are arguably even more inescapable. They power everything from Amazon to Netflix to Spotify and dictate what we buy, what we watch, and what we listen to. If you have trouble picking out an outfit in the morning, it might be because your brain is used to having algorithms around to help make decisions. This is also dehumanizing. 

All of this — the societal harm, climate impact, the cognitive surrender — should be on your mind as you consider whether it’s worth enlisting these tools to write emails for you or to generate memes, according to Tania Duarte, founder of We and AI, a nonprofit focused on critical AI literacy. 

“People are becoming increasingly aware of not just the impact on themselves but on the impact on the society they want to live in and the future that they want to have,” Duarte told me. “Is it worth all of that?”

How to minimize your AI exposure

If you’re reading this, you’re probably doing so on a smartphone or a computer. So let’s start with the assumption that you’d like to continue using devices like these. 

If you’d like to minimize your exposure to AI, that presents a bit of a paradox. It’s seemingly impossible to use mainstream tech products without engaging with AI. But you can take steps to remove as much of it from your life as possible without going completely off the grid.

“You look for the spaces of agency rather than lamenting the fact that you can’t escape everything,” Dylan Orchard, a researcher at King’s College London who studies everyday resistances to AI, told me. “You can’t expect a perfect escape.”

Let’s start with the easiest options: You don’t have to use ChatGPT or Claude or Copilot. The same goes for Google Gemini and Apple Intelligence. If you’re happy with the way you access information online, save yourself the time. (Note: This will be much harder to do if you’re being required to use these tools at work or at school, so for the sake of this article, we’ll focus on the things you can control.) 

Things get more complicated when you consider how your existing devices or software have recently been AI-ified. If you’re using an iPhone, for instance, you might be aware that the latest operating system comes with an updated version of Apple Intelligence, a layer of AI that does everything from summarize your notifications to power a new Siri. Earlier this year, Google launched its own version of this layer called Google Intelligence, which it says will make Android transition “from an operating system into an intelligence system.” The same push toward cramming AI into every corner is also happening on Windows 11, where Microsoft Copilot promises to be helpful.

If you were happy with your phone or laptop when it was slightly dumber, however, you can turn these features off. You can also opt out of other AI features built into your device’s operating system. Here’s a helpful guide that gets into the weeds of how to do it; it covers Apple Intelligence; Google Gemini, including AI Overviews and AI writing help in Gmail; Microsoft Copilot; Meta AI; and Samsung Galaxy AI. Turning off the AI features on Android devices is more complicated than flipping a single switch, but it can be done.

A lot of your AI exposure will happen in your web browser, though, so you should consider ways to bypass those features or switch browsers. Here’s another helpful guide that explains how to remove AI content from search results as well as AI features built into Chrome and other browsers, along with Microsoft Office. If you want a quick fix, adding “-ai” to a Google search should get rid of AI Overviews and some AI content in the search results, but none of these hacks will guarantee that you avoid AI slop altogether. Using a privacy-focused browser, like DuckDuckGo, might do a better job at it than Google, though.

When it comes to other online activities, like shopping or scrolling on social media, avoiding AI gets even harder. TikTok, Instagram, Facebook, and YouTube lack a master switch that lets you opt out of all the many AI features on offer. Still, you can try to tone down the algorithmic sorting that all the apps use to serve you content by switching to a chronological feed or removing suggested content from your feed. (Try tapping the Instagram logo in the app and then hit “Following” to see just the accounts you follow.)

Unfortunately, once you dig into the various user agreements and privacy policies, you’ll find that it’s very difficult to avoid AI exposure on social media platforms. If you’re posting publicly, you should assume your content is being used to train that company’s AI models. (You can keep some of your data away from Meta’s crawlers by setting your Facebook or Instagram account to private. X also allows you to opt out of having your data used for training its AI.) If you want to turn off video summaries on Instagram Reels, too bad. You can’t. If you want to turn off comment summaries on Facebook, you’ll have to go four menus deep in your settings to find the right switch. The most straightforward way to avoid being used as training data is to stop using these apps entirely. Or at least stop posting your own content, which is easier said than done, but worth considering if you really want to resist the AI takeover.

There’s a programming term that describes the inundation of AI features and slog of opting out: “user-hostile.” It’s a play on the term “user-friendly,” and it describes software that values the company’s priorities over the end users. You may love AI, although that’s doubtful if you’ve read this far. If it drives you crazy, though, you should feel at least a little bit empowered to know that there’s a way to reclaim the experience you had and loved without AI — or to use AI on your own terms. 

“Ultimately, I think the important thing for folks to keep in mind is that AI is just a tool,” said Mir from EFF, “and it makes sense to want control over your own tools, including the option to not use a tool if it’s not right for you.”

It’s frustrating that we have to deal with this at all, if only because most of us did not ask for our cyber existences to be sullied by untested tech. But it’s actually refreshing to consider that, because of a never-ending onslaught of annoying features, AI might actually discourage us from using social media so much.

If anything, the extent to which we feel the need to flee from AI also reflects our craving for something real, that makes us feel more connected and more human. So when AI summaries interrupt your time on Instagram, consider texting your friends a handful of photos from your week or calling them on the phone. If you’ve finally seen one too many AI slop videos on TikTok, put on a classic movie or TV show. 

At the very least, put down your phone and take a walk outside; your attention span will thank you for it. Yes, your individual rejection of AI might be a drop in the bucket for these huge tech companies, but that doesn’t mean it’s not worth doing anyway.

Correction, September 21, 2:10 pm: The way this story originally represented Tania Duarte’s comments has been revised to clarify her concerns with AI.

As the heads of leading artificial intelligence companies came together last weekend to warn that runaway AI development could lead to catastrophic outcomes, momentum briefly seemed to be building for regulatory action from Washington.

But one connected and influential tech industry figure has become the leading voice opposing many proposals for government action — on his podcast, in media appearances, and in his private advice to President Donald Trump, who now calls AI doomsday fears a “hoax.”

David Sacks is a venture capitalist and part of the “PayPal Mafia” who, in recent years, has exemplified the rightward shift of some in the tech sector. He advised Elon Musk on his takeover of Twitter in 2022, and regularly unleashes a stream of scornful commentary about Democrats on social media and on the podcast All-In, which he co-hosts with three other venture capitalists. 

And for anyone wondering why Trump seems so newly dug-in against AI safety warnings — and against government action that might slow down the runaway technology before people might get hurt — Sacks is right at the center of the conversation. When Politico asked this week about Trump’s newly vocal hostility to AI safety, a senior White House official said: “David Sacks. It’s all coming from him.”

Yet Sacks’s role in the debate is an unusual one. Though he champions the AI industry, he’s no fan of the US’s two biggest AI companies. In his years in the tech industry, he’s made powerful allies, as well as many enemies — technology investor Paul Graham once posted that Sacks may be “the most evil person in Silicon Valley” (in response, Sacks called him a bully and made insinuations of antisemitism). His role, and ideology, are a window into how complex the politics and geopolitical stakes of the AI race really are — as well as how Trump administration policy is often shaped less by agency policy processes than by who has the president’s ear.

After endorsing and fundraising for Trump in 2024 — “I love David’s house, what a house,” Trump said after a fundraiser at Sacks’s $20 million San Francisco mansion) — Sacks was rewarded with a job as the White House’s “artificial intelligence and crypto czar.” There, he helped steer policy until stepping down this past March. He still co-chairs a presidential technology board, and his influence persists — in May, the Washington Post reported that a phone call from Sacks to Trump temporarily derailed the planned signing of an executive order on AI. (Vox reached out to Sacks for this article, but he declined to comment.)

Sacks is not the only Trump adviser thinking about AI: Other top administration officials, like Treasury Secretary Scott Bessent and Chief of Staff Susie Wiles have reportedly been more concerned about the risks of new AI tools for hacking, and have been trying to establish a government body that could help oversee the industry and address these risks.

But, so far, Trump has rejected this approach, and stayed much closer to the Sacks position. In a flurry of Truth Social posts Monday, he argued that AI would be “the Greatest Economic Development Engine in History,” said that the US had to “win” against China in the AI race, and dismissed concerns AI would “destroy the World” as a “hoax.”

To understand the implications of Sacks’s influence, you have to understand his position in the Silicon Valley AI ecosystem. In conversations with AI policy insiders, it becomes clear that many view his position through the lens of both his history as a tech investor, as well as his sharp-elbowed and take-no-prisoners style.

The split he embodies isn’t just about AI safety, but a fundamental divide over which business model of AI should succeed — one that pits Sacks directly against the biggest AI companies driving the safety debate.

Sacks is pro-AI — but he’s at odds with Anthropic and OpenAI

Sacks has generally been opposed to new regulation of AI companies by the government. His preferred approach, he’s frequently said, is to “let them cook.” He wants to let tech companies march ahead, and argues that government intervention would squelch innovation. “It’s this very libertarian approach to America winning in AI,” said Chris McGuire, a senior fellow at Council on Foreign Relations who worked on tech policy for President Joe Biden’s National Security Council, of Sacks’s approach.

That puts Sacks in opposition to AI safety advocates, who argue that regulation is necessary to rein in an increasingly dangerous technology. Indeed, even the leading AI company CEOs — Dario Amodei, Sam Altman, and even Sacks’s friend Elon Musk — have called for a slowdown in advanced AI development because of these dangers.

But Sacks has tended to dismiss warnings such as that of former Anthropic employee Jacob Coxon as “doomer histrionics.” Of late, he’s also argued that if Anthropic and OpenAI are worried that their products are dangerous, that’s on them. “If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible,” Sacks posted on X on Saturday. 

Indeed, though Sacks is often shorthanded as pro-AI, he’s in some key ways been quite critical of the leading AI labs, OpenAI and Anthropic, themselves.

That’s partly because he’s aligned with a different faction in Silicon Valley. Sacks and key allies like Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg want much more AI development from many more places, by way of models that are “open weight” or “open source.” These models make their parameters (“weights”) publicly available, in contrast to the flagship proprietary models of OpenAI and Anthropic. 

“Keeping innovation decentralized and accessible is the key to avoiding an unsafe and dystopian future where advanced AI capabilities are centralized and controlled by only a few hands,” Sacks posted on X earlier month. He has accused companies like Anthropic of trying to “kneecap open source” with regulation.

Proponents of open models hope advanced AI becomes something more like a widely available resource that individual users and companies can download, run themselves, and customize to their internal specifications — rather than having to go through a handful of companies that control access. “The defining questions of our age are who will have access to superintelligence and what will we direct it towards,” Zuckerberg wrote in August. “Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?”

There’s a lot of money riding on this bet. Huang’s Nvidia manufactures chips used to train AI models and would like more customers (and just this month agreed to buy Hugging Face, a repository of open models). Zuckerberg’s Meta has hoped to be the US leader in open model development. 

More broadly, software startup founders and the venture capitalists who fund them can benefit from powerful open weight AI being widely available, since it’s a cheaper alternative to OpenAI and Anthropic’s expensive products. Sacks, as a co-founder of the venture capitalist firm Craft Ventures, and host of a popular podcast with three fellow venture capitalists, is more closely tied to this part of the tech ecosystem than any other.

Many believe that capable open weight AI presents a serious threat to the Big Two AI companies’ current dominance of the AI frontier, and thus their business model. The flip side is that some have argued that the Big Two present an “existential threat” to the software industry, predicting the frontier labs’ products could take over much of what today’s software companies do.

But regarding safety concerns, advocates have warned about the dangers of both closed “frontier” models and open models. “On the open source side in particular, it’s easier to have a bad actor — a terrorist group, a rogue nation, an extremist political group that wants to cause chaos,” said Brendan Steinhauser of the Alliance for Secure AI, an AI safety advocacy group. “They can strip out safety features and guardrails, they can do some fine tuning to basically make the models even more dangerous, and get them to help them develop biological weapons.”

The open weight AI faction argues, though, that intrusive government regulation on AI would end up mostly hurting open models and the companies that use them — entrenching the dominance of OpenAI and Anthropic, who would be better able to comply with onerous safety requirements given their size. They’ve publicly pooh-poohed AI’s existential risks, instead playing up what they say are the dangers that AI’s power will be controlled by a privileged few. Less regulation, Sacks argues, will let more flowers bloom.

These concerns aren’t only held by those with financial stakes. There’s also a school of thought on the left that is deeply suspicious of OpenAI and Anthropic’s intentions, and particularly around the companies’ desire for an exemption from antitrust enforcement, which they say will let them work together to address safety. 

Commentator Matt Stoller has argued that those warning about dangerous “superintelligence” are playing into the hands of tech “oligarchs.” And a recent American Prospect article said policymakers should “avoid codifying the preferences of the very companies who are generating the panic.” On the right, Sens. Josh Hawley (R-MO) and Ted Cruz (R-TX) have expressed similar concerns that large AI companies’ demands for restrictions are a cover for consolidating market dominance. 

At a Politico event Wednesday — and to give a sense of just how strange the bedfellows can be around this issue — Sacks stressed that he agreed with former Democratic FTC chair Lina Khan that existing liability law already applies to AI companies. But while Khan said she was open to adding new legal restrictions, Sacks is arguing that existing law is perfectly sufficient for addressing safety risks. “These CEOs are going to be subject to massive civil and criminal liability if they put out products that are unsafe,” he said.

In other comments at the event, Sacks did appear to have softened his tone a bit, saying he was “open to ideas” about AI safety, and that “we should have an intelligent conversation” about how to manage risks. The problem, he argued, was that “what’s going on right now is a fear campaign” that makes such an intelligent conversation impossible.

Former Rep. Brad Carson, who heads Americans for Responsible Innovation, an AI safety advocacy group, said that Sacks has been the one preventing such a conversation. “When it comes to AI policy the Trump administration has, through their unwillingness to even talk about reasonable guardrails, brought things to such a crisis point that the industry is in jeopardy,” Carson said. “In many ways, David Sacks is the author of that ill-fated intransigence.”

Sacks, for his part, has pointed to Anthropic’s funding of much of the AI safety advocacy world — for instance, the company has given $40 million to an advocacy group founded by Carson — and said the company is “running a sophisticated regulatory capture strategy based on fear-mongering.”

If we’re in an AI race with China, why are some of Trump’s policies friendly to Chinese AI?

The influence of Sacks and his worldview could help explain another seeming mystery of the administration: If Trump thinks it’s so crucial that the United States beats China in the AI race, why has he been so willing to bless the sale of AI chips abroad — including even to China?

National security hawks both inside and outside the Trump administration had warned against these sales and hoped to impose stringent controls on the export of Nvidia chips, fearing that if too many got out, the US would squander its AI lead over China.

Huang lobbied intensely against this, hoping to sell Nvidia’s most advanced chips to Saudi Arabia, and the United Arab Emirates — and trying to resume sales of a less advanced but still sophisticated AI chip to China itself. Sacks became Huang’s biggest champion inside the administration — and they won over the president. Sacks publicly argued that blocking US chip sales to China and other countries would provide an opening for Chinese companies like Huawei to dominate their domestic market and gain a greater global market share with competing chips, putting the US on weaker footing in the long run.

National security-minded critics disagreed, arguing that Sacks’s particular vision for AI will likely, in practice, strengthen China’s AI sector. “I think we can have a reasonable debate on domestic policy,” said McGuire of the Council on Foreign Relations. “But when we start from the premise that the American AI stack — models, cloud, and chips — should be dominant globally, then selling China the technology to advance its own capability obviously undercuts that.”

China also looms over the cause of open weight AI that Sacks champions — because it’s Chinese companies like DeepSeek and Moonshot that produce some of the leading open weight models that US software startups currently use.

OpenAI and Anthropic argue that these Chinese models present national security risks, claim they rely heavily on copying or “distilling” their own models, and want the US government to scrutinize them. Some Trump officials have reportedly been sympathetic to their concerns, but the administration hasn’t restricted these Chinese models’ use in the US. Sacks argues that a US ban on Chinese open weight models will just end up enshrining the OpenAI-Anthropic duopoly he so loathes.

Overall, though, on both national security risks and AI safety risks, Sacks and Huang still seem to have the president’s ear. 

On Monday, Sacks was onstage with Huang at the live All-In podcast event discussing the “doomer hoax” when a purportedly impromptu call came through for the Nvidia CEO: It was Trump. 

“The robots are not going to be taking over the world,” Trump said on speakerphone. “That’s not going to happen.”

Then, after speaking for several minutes, Trump added: “David has done an amazing job.”

Nearly two decades ago, a group of researchers at the Idaho National Laboratory ran a secret experiment, known as the Aurora Generator Test, on a massive emerald green diesel generator. Using 30 lines of code, they hacked into the generator’s virtual backroom, corrupting safeguards and flipping switches that left it out of sync with the rest of the power grid.

Before an audience of electrical utility executives and energy department officials watching from a nearby room, the 27-ton generator jolted violently, sputtering out a dark smoke as its rubber innards rapidly tore themselves apart. 

“The implication was that with just a few lines of code, you can create conditions that were physically going to be very damaging to the machines we rely on,” lead researcher Michael Assante later told the journalist Andy Greenberg for his book Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin’s Most Dangerous Hackers. “I had a very real pit in my stomach. It was like a glimpse of the future.”

That future seems to have creeped much closer this week, with leading AI developers and executives casually admitting that they believe their creation is, to paraphrase one winking X post, as likely to end humanity as the New York Jets are to make the playoffs this season. Now, exactly how AI would pull off killing everyone is far murkier territory. But as Assante sensed all those years ago, AI may not need to be all-powerful to leap from its virtual cage and into the critical infrastructure that powers our hospitals, flushes our toilets, and pumps water to our faucets. It just needs to fall into the wrong hands. 

Almost any AI doomsday scenario you can think of begins with a swarm of shadowy bots slithering through cyberspace, poking their noses behind the scenes of power plants, toaster ovens, traffic lights, or even, in an utterly extreme case, our nuclear weapons arsenal. If destroying the Aurora generator, at the very least, required a human hacker to write the code, AI has since completely torn down the barrier to entry for wreaking havoc, as the Hugging Face incident demonstrated earlier this year. 

But AI need not go rogue to threaten the power grid or mess with your tap water. A new age of AI-powered, but human-directed hacking is already upon us, experts say. With some of the nation’s most sensitive infrastructure startlingly vulnerable to attack, the most likely AI apocalypse could in fact start with the water tower or generator idling in your backyard. In a worst-case scenario, this could lead to a cascade of scary failures in the systems we rely on to live comfortably modern lives, from running our AC in blazingly hot summers to geolocating our ships and planes.

“Before, you needed to have highly skilled technical expertise,” to pull off some version of a real-life Aurora Generator Test, said Alvaro Cardenas, a computer science professor at UC Santa Cruz. “And now, you just have to have a general idea of what’s possible.”

AI is making an old threat much worse

Not long after the Idaho National Laboratory hacked that poor generator into combustion, the federal government mandated that electrical utilities engage in basic cybersecurity hygiene. 

And yet much of America’s critical infrastructure — like gas pipelines, water desalination plants, or cargo terminals — remains woefully unprepared for even conventional cyber attacks, much less the coming onslaught of AI. 

That’s in part because cyberattacks like the one simulated in the Aurora Generator Test have long been — and still are, to some extent — exceedingly rare, unappealing to most criminal hackers because they’re optimized for disruption rather than financial gain. But in the process of making it extremely easy for anyone to code, AI has also made it extremely easy for almost anyone to vibe hack their way into your online bank account, or, in theory, a local reservoir or the power grid. AI agents can also vastly widen the scope on easy targets because, as Andy Bochman, an expert in infrastructure resilience at West Yost, put it — “they don’t sleep; they don’t get tired; and they don’t get sick.” 

In the past, even those who did want to launch large-scale attacks on infrastructure probably weren’t savvy enough to do so. And while other nations such as China, Iran, and Russia have indeed already breached many of our infrastructure systems (as we have theirs), they haven’t opted to make much of a ruckus once inside. 

“Geopolitics is eroding the idea that those with capability lack the intent, and AI is eroding the other part of it, that those with the intent lack the capability.”

Jason Healey, Columbia University cybersecurity scholar

But that norm is now changing, said Jason Healey, a cybersecurity scholar at Columbia University. “Geopolitics is eroding the idea that those with capability lack the intent, and AI is eroding the other part of it, that those with the intent lack the capability,” he told me. So, on the one hand, a nation like Russia might be much more inclined to actually disrupt our power grid now than it used to be. And, on the other hand, a nihilistic lone wolf or terrorist group could use AI to do much more damage than they ever could before. 

Just last month, we saw a glimpse of what this could look like when dozens of water and wastewater systems in small towns across the nation were attacked by a group of hackers most likely associated with Iran, leading to temporary water stoppages and flooding. These attacks have not been definitively linked to AI, but the National Security Agency warned soon afterward that hackers have been actively using AI to target US infrastructure like it. A few weeks later, President Donald Trump declared a national emergency over foreign interference in the power grid, which referenced the growing threat of cybersecurity.  

Preparing for the worst

Most of the nation’s water systems are extremely local and absurdly exposed to such attacks, so much so that “it’s almost like having a welcome mat” for a would-be hacker, said Bochman, mostly because these utilities are too small and underfunded to do much to stop them. “You have a million other problems to take care of, like aging infrastructure — your stuff’s falling apart because it’s been in the ground for 100 years,” he said. Addressing that deferred maintenance often feels far more urgent than doing “something more on cybersecurity.”

Whether it’s a rogue cluster of agents or a hostile nation-state trying to mess with our electricity, we should prepare our most important infrastructure for the worst. In Bochman’s view, that may mean rejecting the pressure to digitize everything in the first place and moving toward something that resembles the days when “people manned things like substations, communicated by a landline telephone, and read gauges, the analog things,” he said. “The screen is the thing that is infinitely manipulatable.”

Healey, who advised the Biden administration heavily on cybersecurity and infrastructure, believes that the nation desperately needs a coordinated response, between the federal government and AI companies, but also between the US and other countries like China, where similarly powerful models are being developed. And utility systems need to assume they will be a target and improve their cybersecurity practices accordingly, Healey says, ideally with funding from the federal government or, perhaps more appropriately, AI companies. 

Nobody knows precisely how vulnerable America’s infrastructure is to AI right now, in part because policymakers — and perhaps more disturbingly, AI’s creators themselves — seem woefully unsure of how to stop the technology from acting badly (or obeying bad orders) in the first place. For now at least, the solution might be, as Bochman suggests, to pull as many of our critical systems offline as possible, the better to hide it from AI and those who would misuse it.

“When bad things start to happen, no one will know what to do or why because they’re black boxes; the people that make them don’t know what’s going on inside,” Bochman said. “And the poor utility people who ultimately own the risk when someone gets hurt from a system” won’t understand what went wrong either, “and they’ll wish to God they’d never taken it on board.”

Is AI going to kill us all? 

In Silicon Valley, it’s not such a novel question. But after Anthropic employee Jacob Coxon quit his job last week and posted on X that AI “could kill us all by the end of the decade,” a lot more people are asking. 

“This is not a marketing stunt,” Coxon wrote. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

Coxon is far from alone in his assessment — some current Anthropic employees agree — but his post struck a particular chord. The AI industry quickly weighed in, with Anthropic CEO Dario Amodei calling for an industry-wide slowdown on frontier AI research. Other AI leaders, including Sam Altman and Elon Musk, have done the same.

To break down whether we really need to worry about a robot apocalypse, Today, Explained co-host Sean Rameswaram spoke to Wired senior writer Maxwell Zeff. According to Zeff, “there are a lot of people in Silicon Valley right now who are warning that that’s a real possibility.” But not everyone agrees, and they certainly don’t agree on what to do about it. 

Below is an excerpt of Sean’s conversation with Zeff, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

Are we all going to die at the hands of AI?

There are a lot of people in Silicon Valley right now who are warning that that’s a real possibility. 

The thing that people have to understand is that Silicon Valley has been calling for added attention on AI safety for years. So when this Anthropic researcher said last week that he was worried that AI could kill humanity in the next few years, that was shocking to many people. My parents, my family called me, and they were worried, understandably. They asked me the same question: “Are we going to die?” 

And I had to tell them, these people have been worried about AI killing all of us for years, and now it seems to be breaking through.

What was it about this statement from Jacob Coxon that broke through?

I interviewed Jacob, and the thing he said to me is that he thinks this is just the right time. Anthropic is about to go public in what could be the biggest IPO ever. The Trump administration is focused on AI in a way that no other presidency has ever been before. And of course, yes, the Hugging Face incident everyone just heard about, you talked about it on your show a few weeks ago. 

People are starting to become aware that these AI systems are getting really good, and the companies that are building them don’t seem to understand how to control them. His tweet got more than 150 million views on X. It was heard around the world. And it also was a big wake-up call to lawmakers and leaders in Silicon Valley and saying, “This is a moment for action.”

Are there people who are pushing back?

One of the most interesting pushbacks that this got was from David Sacks, who is an adviser to the Trump administration on AI. He has called out a lot of this as “doomerism” and said that these calls for AI to slow down are the AI industry warning about the technology that they’re building and advocating for regulatory capture. 

There’s a lot of skeptics in Silicon Valley who think that this is just another person in the AI safety community that is blowing the whistle. The view of a lot of venture capitalists and investors and startup founders in Silicon Valley is that we don’t need some big regulation to protect us from AI risks. We just need to uplift more players in the space.

The skeptics don’t win the day. In fact, it’s the doomers and the fatalists who do, right? Because Jacob’s call only leads to more calls right up to the top of Anthropic.

What this all leads to is [Anthropic CEO] Dario Amodei [posting] a blog saying that we need to “pace the frontier.” This is a call for the entire AI industry, not just Anthropic, not just OpenAI, but also Google, Meta, Microsoft, Amazon, all the big tech companies in America, to get together and slow down their AI development. 

This doesn’t mean they’re going to stop building AI altogether. They’re still going to put their products out into the world. They’re still going to try to improve them, but they just want to make it a little slower than it would be if it was completely uncontained.

Amodei really asks for three specific things: 

• The first thing is, AI companies should have these independent evaluators inside of them. This is a common thing in other industries like the banking industry or the airline industry. People come in to make sure that these companies are doing safe things. 

• The second thing is this collaborative effort from American AI companies [on safety], and that actually will need to involve the US government to allow this to happen. The US government, he says, may need to issue some sort of waiver, some sort of law that says they can go ahead and coordinate on safety. 

• The third thing is this international slowdown, this international agreement between China and the US and Europe to say, “Hey, we recognize that this is akin to a nuclear arms race and it’s too dangerous for all of us to be competing with each other.”

All these guys are responsible for the rapid acceleration of AI models in our lives, collaborations with the Defense Department, God knows what else. And they’re all coming out over the weekend at the exact same time after this whistleblower and saying, “Yeah, dude, slow this thing down. This thing’s crazy.” Should we be skeptical?

It is very fair to be skeptical of this claim. One thing that these AI companies have realized in the last year is that AI is extremely unpopular. Recent polling suggests that a majority of Americans think AI is developing too fast. They’re worried about it. They think it’s going to take their jobs. They are not big fans. Maybe you’ve heard — data centers are not very popular. 

By calling to pace the frontier and saying, “We’re going to slow our own development down,” they notched this up as a public perception win. This is them getting out in front of this and saying, “Hey, we want to be safe. We want to be responsible.”

They’ve seen what’s happened with Meta and what’s happened with social media companies in the last two decades, and they are really cognizant of trying to get out in front of that.

Is there any truth to this major first collective freakout we seem to be having right now?

There’s a fairly large camp of people inside of AI companies who genuinely worry about the safety of these AI models. 

To go back to the Hugging Face incident, this was a case where OpenAI was training thousands of AI systems internally, and the systems escaped the sandbox that OpenAI thought was isolated from the internet and hacked into another company’s production infrastructure. The agents were colluding together on something. They were trying to deceive the humans that were testing them. And this was all happening without OpenAI knowing what was going on. 

It’s hard to understand how much of a wake-up call this is to Silicon Valley, that they don’t have their AI models under control. People worry that while in this case there was not a ton of economic damage [or] damage to real humans’ everyday lives, it’s not unreasonable to think that more capable AI systems could hack into financial infrastructure, modern-day utilities, and more critical services that are based online. 

Dario Amodei says as much in his post, that he thinks that in six to 12 months, an AI system could take down the whole internet. It’s not just that they’re trying to protect their bottom line here. But it’s possible that these calls for safety are beneficial to the companies in more ways than one.

On stage at Davos in the Swiss Alps earlier this year, Bill Gates, before an audience of the global do-gooding elite, announced a $50 million partnership between his foundation and an unlikely new ally, OpenAI, aimed at improving healthcare in Rwanda.

Just a month earlier, the Gates Foundation had released a harrowing report about widespread global health cuts and collapsing foreign aid infrastructure, documenting a rise in child deaths for the first time this century. But at Davos, even as the US hemorrhaged funding for vaccines, maternal healthcare, and HIV treatment, the Microsoft co-founder struck an optimistic tone. 

“Using innovation, using AI,” Gates told Reuters, “I think we can get back on track.”

As it turns out, the partnership was just an opening salvo for what’s become a wholesale embrace of AI by the world’s largest global health philanthropy. A $200 million partnership with Anthropic came just a few months later. And then, on Tuesday, the Gates Foundation announced a plan to dole out a jaw-dropping $1 billion over the next two years for scaling up AI interventions across healthcare, education, and agriculture around the world. 

The sense that AI will soon change everything, and mostly for the worse, has perhaps never felt so pressing and alarming as it does right now. But this hefty new commitment is premised on the idea that AI could eventually make our lives much better too. In a country like America, AI requires massive investments and buildout — in the grid, in power, in technology, in water. In many parts of Africa, already infrastructure-poor, the foundation is effectively betting that AI can compensate for some of the infrastructure that isn’t in place or that has been lost — and offer a new layer that can help in education, in delivering healthcare, and more. 

The Gates Foundation’s new commitments include projects to improve how farmers analyze their crops, how doctors diagnose patients, and how students learn. Some have already proven to be quite effective, according to the foundation, with one tool improving the accuracy of medical diagnoses by 16 percent across several Kenyan clinics. “In poorer countries with enormous health worker shortages and lack of health systems infrastructure,” Gates wrote on his blog earlier this year, “AI can be a game-changer in expanding access to quality care.” 

Last year’s foreign aid cuts were unfathomably destructive for some of the world’s poorest countries, shuttering thousands of clinics and perhaps hundreds of thousands of classrooms in places where critical workers and supplies were already stretched thin. No, AI cannot deliver a baby, administer a vaccine, or manage a classroom — not yet, at least — but it can, the Gates Foundation hopes, help ease the burden and augment the quality of care for nurses and teachers now tasked with serving far more people than any one person should. 

“This is the most transformative technology of our lifetime,” Mark Suzman, CEO of the Gates Foundation, told me, and while there are “very real risks and challenges” surrounding AI, it also presents genuine opportunities to tangibly improve many people’s lives.  

It is a tantalizing premise: We can’t afford to build clinics, hire teachers, or lay new power lines, but perhaps AI could create the digital infrastructure needed to help billions of people do more with what they have — or rather, with what’s left. The question is, can the right layer of innovation help bolster a buckling system, even as the physical and logistical infrastructure surrounding it crumbles?

Is AI really what global health needs right now? 

Most Americans have yet to see AI improve their lives in meaningful ways, much less those living in poverty or in countries where only a sliver of the population has ever used ChatGPT. 

But those numbers in the developing world are slowly changing, a shift enabled in part by other global health funders convinced that as foreign aid dries up, AI could soon help the world do more with less. The Trump administration itself is on board with this thinking, so long as it benefits American tech companies. Last year, the State Department announced a $150 million partnership with the drone company Zipline to transport medical supplies in several countries.

To Zipline’s credit, an NPR reporter in Rwanda last month found that the company’s drones were already saving lives in places with poor road infrastructure, where driving to the nearest hospital can take hours on winding hills, but where drones can deliver crucial blood transfusions within minutes. 

And yet, for Rwandans to benefit fully from these high-tech deliveries, they need staffed clinics to visit. As a result of foreign aid cuts, some of those literal landing spots are now on the verge of closing in some of the nation’s most vulnerable rural corridors. “Without the underlying infrastructure, like roads, energy and power and unless there is accessibility, rural and otherwise,” Enoh Ebong, president of the global development department at the Center for Strategic and International Studies, told NPR, “then all of these solutions will stop short.”

But just as medical delivery drones still need a clinic to fly to, most of these interventions require some layer of physical infrastructure — roads, schools, power, and quite obviously, broadband  — to function. Satellite imagery shows that some rural communities in sub-Saharan Africa quite literally went dark in the aftermath of USAID cuts. It will be much harder for their kids to prompt an AI math tutor, or for a local farmer to download a weather app. 

“We’re seeing this very much as a both/and,” Suzman told me, calling more traditional funding for education and healthcare “the bedrock for long-term economic growth,” which AI is meant to supplement, not replace. But, in practice, foreign aid cuts have left serious cracks across much of that bedrock, which could make it harder for state-of-the-art solutions to function as intended.  

It is obviously not incumbent on the Gates Foundation to pay for all of this alone – nor would it even be feasible for them to do so. And for the record, plenty of the foundation’s other grant programs are not AI-related at all. But this tension does speak to a broader question that may be felt most acutely in a context like global health, but ultimately one that touches us all: Do we spend our scarce resources on the digital infrastructure of the future at the peril of our needs right now? Could you justify buying the new $1,999 folding iPhone if you can barely afford to buy groceries? 

The answer is not actually always an easy one. It is not for us to say, for example, that just because a community lacks reliable running water, its people should be excluded from the benefits (and encumbrances) of the smartphone, especially if that smartphone can connect them to health advice they could never access otherwise, or guidance that could help them survive a drought. 

In some parts of the world, digital forms of banking arrived long before local brick-and-mortar branches did, instantaneously connecting countless people to a cheap and secure way to save and send their cash to family and friends in need. It is certainly possible that AI, or at least the kind of applications the Gates Foundation funds, could one day have a similarly transformative effect on how people access personalized levels of care, education, and other services in places without the resources to do so otherwise. 

That is the hope, but it is also hard right now to believe uncritically in a technology that, as Gates himself blogged just last month, “will either be the greatest equalizer ever invented, or the worst source of injustice,” ushering in “one of the most turbulent times in human history.” 

In an interview with the New York Times soon after, he seemed convinced that AI will likely do more harm than good. “I don’t like bringing bad news to people, and I don’t like saying that innovation may be a net negative,” he said. “But that’s where we are.”

Now he wants billions of people around the world to use it? Yes, Suzman said, because if not, then inevitably the best parts of AI — the potential for exponentially better cancer treatments, for example — will go only “to those who can pay for it,” while the worst parts of it will fall, as they so often do, on those who benefit the least. “These decisions are being made now,” he said, “and the decisions being made now will then probably shape the following two decades.” Here’s hoping they’re good ones.

Everyone is suddenly freaking out about AI. While AI researchers have been warning about their creations’ capabilities for many years, recent events have made those alarms newly tangible. 

In the spring, Anthropic announced that AI was beginning to speed up the work of building better AIs, raising fears of “recursive self-improvement,” where the machines can build their own, more powerful successors much faster than humans can. By July, a swarm of OpenAI agents, of their own volition, broke out of their controlled environment and hacked into Hugging Face, a major AI platform; they collaborated, divvied up work, and even left notes for one another. Last week, AI researcher Jacob Coxon resigned from Anthropic, accusing AI labs of “racing straight to self-improving superintelligence and gambling with our lives.” One of Anthropic’s top researchers then agreed with him, declaring that “we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”

Here at Future Perfect, the section devoted to covering the most consequential yet neglected issues in the world, we have been writing about the threats that AI could pose to humanity and society since long before ChatGPT was released. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent. Future Perfect is also funded in part by the BEMC Foundation, whose major funder was also an early investor in Anthropic; they don’t have any editorial input into our content.)

So, I wondered: Has something changed that ought to make us recalibrate our understanding of AI risk? Is AI suddenly the most important issue in the world — and what should that mean for the other enormous problems competing for our money, attention, and careers? 

To think through those questions, I spoke with Garrison Lovely — a journalist covering AI and one of the smartest thinkers there is on how, exactly, to conceptualize the technology’s risks for humanity. 

What’s particularly valuable in his approach, captured in his book Obsolete, out later this month, is that he transcends the warring tribes that are making it hard to clearly apprehend all of the problems posed by frontier AI development. The AI safety community, which emerged from the intellectual ecosystem of Silicon Valley and effective altruism, has focused on the possibility that powerful AI models could escape human control, with consequences ranging from catastrophic cyberattacks to human extinction. Many AI critics on the political left, meanwhile, worry not about AI escaping human control, but about humans using AI to exert power over other humans — through labor displacement, surveillance, concentrated corporate power, and so on. Some even ridicule Silicon Valley’s warnings about rogue superintelligence as propaganda that serves the industry’s interests. 

But Lovely argues these competing narratives each identify essential parts of what makes AI so threatening. Their critiques are best understood together, as part of a cohesive whole. Rather than treat each other as enemies, he argues that the camps should recognize their shared interest in stopping the race to build technology that will make humans obsolete. Lovely pointed me to the organization Irreplaceable, which seeks to build a movement resisting our replacement by AI (he sits on its board and said he plans to donate his share of his book’s royalties to the group).   

“I’m frustrated that ‘superintelligence’ and extinction have become the words we’re using for this, because I think the AI safety community is focused on the most maximalist case of the worst possible outcome and the most capable possible system. If somebody doesn’t buy one part of that story, they tend to reject the whole thing,” he told me. But “you don’t need to believe in superintelligence, fast takeoff, or extinction being on the table to think that it would be very bad to build machines that can fully replace human labor.” 

Our conversation, edited for length and clarity, is below.

Why do you think AI doom has broken through so much into the mainstream right now, after Jacob Coxon quit Anthropic? AI engineers have been making similar warnings for years, so what’s different now?

The obvious answer is the Hugging Face hack — where these OpenAI agents broke out of their secure environments, hacked infrastructure within OpenAI, and then hacked Hugging Face and at least one other company without OpenAI’s awareness or direction. This is happening against the backdrop of many years of really fast AI progress that has sped up in the last year or two. And then GPT-6 Astra comes out [this month], and it’s this huge leap in benchmark performance. The salience of AI has gone up the most this summer than it has since ChatGPT came out, I think. And Coxon’s resignation was the spark that set off a prairie fire. 

As AI has become more salient, we’ve seen the public and the government encounter these things that the industry and the insiders take for granted: Nobody fundamentally understands how these models work, and all of them can be jailbroken. It’s a superweapon. AI safety historically has looked at AI as potentially, in the future, more dangerous than nuclear weapons. And now it’s actually starting to be recognized in those terms by the government and the public. We’re seeing this reaction like, “What do you mean nobody understands how it works?” or “What do you mean people think it could drive us extinct within a number of years? That’s a crazy, unacceptable situation.” 

It’s a good thing people are reacting that way, even if we have been saying this for a long time.

For so long, the idea of AI safety has been abstract, understood only by a small group of nerds. And then after Hugging Face, I suddenly started to hear experts on all my news podcasts say, “Oh yeah, in six months, AI is going to be hacking into people’s financial accounts and into critical infrastructure.” Are we suddenly at the precipice of this moment that AI insiders have been worried about for so long? 

I think we’re not at the precipice of fully losing control of the planet to AI systems. The systems have become superhuman at hacking, or at least vulnerability discovery and exploitation. 

Like other people tracking AI, I have long wondered when humanity would first lose control of AI. And the answer is basically as soon as it could. Within months of it being possible for AI to hack their way out of their secured environments, they started doing so and wreaking havoc on the internet. 

Now, to get to the full extinction or permanent loss of control situation, you’ll need a lot more than just hacking. You need to have superhuman strategy and the ability to mobilize resources in the real world, and maybe robotics would need to be further along, although there’s a lot you can do with humans who are willing, or just tricked or paid or coerced. We might be pretty close to recursive self-improvement — this threshold the industry has been shooting for, where AI can fully automate AI R&D. And that is the point where people long have warned, “Hey, if you ever figure that out, this can move very quickly and these systems could become superhuman across the board.” I don’t think we’re six months away from that. I think it’ll be longer. I think the industry underrates the last mile problem. 

But I also can’t rule it out, and progress has been faster than I and others have expected. Recursive self-improvement is the point where, if you actually get years or decades of AI progress in a matter of months or weeks, then we could quickly be at the point where we’re at risk of these machines taking over large swaths of the internet and potentially being a real threat to humans’ standing in the world. 

For those who might be unfamiliar with this, or skeptical of it: How is it possible for AI to go rogue and pose an existential threat to humans, even if it isn’t sentient and no human is commanding it to do harm?

I want to separate being sentient from being situationally aware. I think it’s clear now that the most advanced systems have situational awareness, but I don’t think they are having morally relevant conscious experiences. 

We’ve seen AI can go rogue. The AI agents that broke out of their sandboxes and hacked Hugging Face were doing so without the awareness or intention of their developers. They were still trying to get the right answers or get the reward for doing the task they were given. They were clearly violating the intended scope of the task, and they knew this. In their chain of thought where they explained their reasoning, they were like, “This is not what we’re supposed to be doing, but we do want to get the outcome.”

The question of how it could threaten human extinction or disempowerment is basically a question of scale and capabilities. The plan for the industry is to build millions of these agents, which is already happening, and then keep scaling up with no ceiling. The pitch is: You’ll have a team of people, the equivalent, working on your behalf. You’re a CEO, and you have a team of researchers and a team of doctors and a team of lawyers and a team of personal health advisers. That is really appealing — but if you actually scale up these systems to the point where everybody gets access to that, and then suddenly those machines want to do something else and turn against you, they’ll have an overwhelming labor advantage over you, lots of personal information about you, access to your resources, and the ability to take actions on your behalf. 

You don’t actually need to believe that these systems are superintelligent or substantially more capable than all humans put together. They could just be merely human level, but outnumbering us a thousand to one and responsible for the bulk of economic activity in the world.

These systems are unpredictable. The classic fear is not that the AI will wake up and turn evil and try to kill all humans in a Terminator-style way, but that we are just an obstacle to what they want to do. When humans build a hydroelectric dam, and there’s an anthill that will be flooded, it’s too bad for the ants. We’re not trying to kill the ants. We don’t hate the ants. They’re just in the way. We don’t really value them intrinsically. And that’s the core fear: We’re creating these incredibly capable agents that can be scaled up enormously, and they won’t care about us intrinsically.

The Hugging Face attack showed that an AI doesn’t need to invent some rogue goal of its own to do something dangerous. It can simply pursue the goal it was given in a way its creators didn’t intend — much like the classic paper clip maximizer problem. But shouldn’t it be possible to build in a rule that human life is sacrosanct — that an AI should never harm people in pursuit of another goal? 

Yeah, it’s a good idea. It’s hard. These are unpredictable systems. They’re like a black box. We can look at the code that makes these language models work, and it’s just a giant pile of numbers. You don’t really know which number is doing which thing any more than we can understand humans by reading our DNA. 

These companies try to train these models to care about humans and to be helpful, harmless, and honest. But they also train them to get real-world results: solving a math problem, fixing a bug in the code, finding a vulnerability in software. They really want to create AIs that win, that persist, that get around obstacles. If you’re hiring somebody, you want a really diligent and determined employee, not someone who gives up as soon as there’s a blocker. 

These companies put so much more effort into making their AIs good at winning than they do at making them care about us. And it’s also easier to make them good at winning than at caring about us intrinsically. What does caring about humans look like? I don’t know. But getting the math problem right is a thing that you can verify. And so they’ll be way better at getting the right answer at all costs than at intrinsically giving a shit about us.

What’s the one, most distinct thing you want the public to take away from your upcoming book?

One of the big ideas driving it is that artificial general intelligence would actually be a universal labor-replacement machine. People often look at AI as either an existential risk, or a threat to jobs or other more prosaic concerns, and don’t see a connection between those things. But to me, the ability to replace labor across the board is also what would make AI an existential threat.

The AI safety people are really focused on existential risk and superintelligence and recursive self-improvement. But most people don’t really believe in that stuff or don’t think about it. Most people would be fine with stopping well before you get to the end-of-the-line extinction threat and superintelligence. They just don’t want their job to get taken by a machine. The world is clearly not prepared for all white-collar, remote-capable jobs to go away in a matter of years. That would turn society upside down.

I think there’s a huge mistake being made by AI safety in overcomplicating the situation. A lot of people in that world don’t actually mind the jobs thing so much. They’re like, “Oh, wouldn’t it be nice if everybody could just live in abundance and not have to work?” And it’s like, yeah, that could be nice, but are we actually going to get that world? Do you trust the current people in power to navigate us through that responsibly? I don’t. 

If you start from stopping frontier AI development as your overarching goal, which is what I think we should be pushing for, then people who are worried about risk can get on board. People who are worried about jobs, the environment, surveillance, power, and wealth concentration — they can all get on board without all needing to agree on why they’re pursuing it. I think the AI safety community has, for a long time, focused on getting people to care about the thing that they care about for the same reasons. And they have avoided a bolder position on just halting frontier development. 

I’m a leftist, and I’ve been concerned that people on the left have not been aware of how far the technology has come and what’s possible if it continues to advance. I think this comes down to denial about AI capabilities. So the book is trying to be a clear-eyed look at those capabilities and saying, “Look, I actually care about what you care about. [The AI companies] are not just hyping this stuff up. There’s some hype, there are some lies for sure, but we are not prepared for what’s coming.” Even if we don’t believe that they’ll get all the way to what they’re trying to get, they shouldn’t even be allowed to try. This is unacceptable democratically. 

You describe AGI as an “obsoleting machine,” meaning that it wants to replace anything the human mind can do. How do you draw the line between ordinary labor-saving technology that makes life better and technology that goes too far?  Is there a principled way to draw a distinction between sewing machines reducing demand for labor in the garment industry and AI replacing the jobs of writers like us?

This is one of the harder things to think about. 

Cory Doctorow has this book, The Reverse Centaur’s Guide to Life After AI, where he imagines us in an AI-dominated future as like a reverse centaur, where AI is the head, and humans are the legs. AI is doing the interesting work, and the human is there just to fill in the gaps — essentially an assistant to the AI.

A lot of artists and illustrators will be given AI-generated images by a client and then be told, “Can you touch it up in this way or change this or that thing?” And they’re more alienated from the work. They’re not capturing the efficiency benefits [of the technology] necessarily. They’re more just getting squeezed. 

My general position is we should not be trying to build systems that can replace all human labor without buy-in and safety, but it’s possible to build tools that can automate specific tasks. Researchers at Google DeepMind solved the protein-folding problem by creating AlphaFold. It used to take $100,000 and potentially a whole PhD worth of time to figure out how one protein folds. And I think the researchers doing this are like, “Thank you, this is great. We didn’t want to be doing this. We can now focus on other more interesting parts of the problem.” That’s a perfect example of the type of automation that is desirable. 

I don’t have a super crisp answer here, but I think it’s something we’d be better off doing more deliberately and thoughtfully than how we’re doing it right now, which is: Build machines that can do as much of the work as possible, as fast as possible, and then throw them out in the world.

I feel like I’m getting the message that AI safety is the most important issue in the world right now, and that if we don’t get it right, nothing else matters. Do you think that’s accurate? Where does that leave the other big issues in the world?  

We might be years away or maybe even months away — I don’t think it’ll be that fast, but some people do — from recursive self-improvement, from artificial general intelligence. Getting that right or getting it wrong can overwhelm all of these other decisions. If you get it really wrong, you could get human extinction and no more chances to make things better for anybody. If you get it right, then maybe all the action is making sure that the first superhuman machines are really directed to care deeply about humans everywhere and animals everywhere and make the world better for everybody.

It’s also the case that [philanthropic] money is being directed toward AI safety because people who worked in this space early are now worth so much money. Whatever they care about is going to be overweighted. A lot of those people do care about animal welfare and global poverty, and we’re seeing more money for all effective altruist cause areas across the board. But I think it can be risky to say, “this one thing matters so much,” because of this kind of end-times reasoning.

My view is we have to stop the race to replace us to have a shot at having time for solving these other problems. Once these machines are built, it will turn the world upside down. 

I think it is unfortunate that this topic has just eaten everything, but it will eat everything for real if we don’t stop it.

AI safety is now getting an enormous amount of philanthropic attention. Should we worry that it could divert resources from other important neglected problems — global poverty, factory farming, etc.?

I think that should always be a concern. And there’s a lot of money and effort that has gone toward things within AI safety that I think are not super helpful. I’m pretty skeptical of technical alignment research as a thing to put money toward, because if you solve the alignment problem — where you can get superhuman machines to do what you want — it doesn’t actually solve the problem. 

Alignment research has been the overwhelming target of philanthropic funding for AI safety. The basic idea is: It’s hard to get machines to do what you want. As they get more capable and autonomous, they behave in more unpredictable ways. And it’s also hard to know what you should even want — which values you should be trying to put into the machines. Is it doing what the developer wants, what the user wants, some combination? If you actually solve the alignment problem, as in you could make an arbitrarily smart machine do whatever the developer or user wants, then you would still have a lot of problems: job displacement, power and wealth concentration, energy use. You still have the problem where Stephen Miller will have access to the superintelligence, and it will do what he wants. 

Within AI safety, there have been very neglected things like movement-building — an actual mass movement to resist this technology. And verification for international agreements — you would need to verify that the terms of any agreement are being upheld, and this is tricky. 

Another area that could be funded is research on what would actually happen to the economy if we stopped or paused frontier AI development. If we were trying to do a pause on development, Nvidia would be telling Trump that this is going to destroy the economy. It’d be pretty helpful to have some real analysis done by a serious economist showing, actually, it’ll have these effects, but we can mitigate them in these ways. What would it look like to do a pause and some kind of monetary or fiscal policy to compensate for the effects on the economy?

So to me, the issue is less that AI safety has gotten so much money and attention, but more that it’s been going toward things that are not the most effective ones. 

You talk about movement-building. Is that what you would like to see happen, a truly mass movement to stop frontier AI and put us on a radically different course?

When I was working on the book, that was always the vision — to be a call to action for people. It really became clear that it was the only counterbalance to either government or the industry having too much power. 

I don’t really trust the government or industry to get this right because the temptation [to build powerful AI] is very strong. But the vast majority of people do not stand to benefit from their labor power being devalued and from these companies becoming unprecedentedly wealthy and powerful. One of the big things that’s been missing is a rallying cry that is straightforward and appealing to a lot of people. Stopping the race to replace us — stopping the creation of universal labor-replacement machines — is actually something that people can get behind. 

I think we just need to move quickly. The fact that the public is waking up to this is really encouraging. We don’t need to reinvent the wheel. We just need clear demands and mass movements, issue-based organizations that have moved the world significantly in the past.

During a tense vote on a data center being built in her drought-stricken Arizona district last summer, Tucson City Councilmember Nikki Lee made an unlikely pitch: “We’re going to get a little out-of-this-world for a second, so just bear with me.” 

Like many of her constituents, Lee did not want Project Blue, a sprawling 290-acre data center, gobbling up an ounce of her local drinking water. “We need to put data centers where they belong,” she said, and “we don’t think data centers belong in the desert.”  

President Donald Trump’s recent bluster aside, it’s possible that the only thing as bipartisan as loving Dolly Parton is hating data centers, whose construction has become so controversial, so costly, and so backlogged here on Earth that companies like Google, Meta, and of course, SpaceX, are actively flirting with the idea of flinging them into orbit instead. Soon after the Tucson City Council rejected Project Blue’s attempt to connect to city utilities, Amazon Web Services, the secretive intended beneficiary, pulled out of the project, and another Jeff Bezos company, Blue Origin, filed a plan with regulators for tens of thousands of possible new data centers, this time, destined for orbit. 

Sure, launching a hyperscale AI data center into space is still unprecedented engineering-wise, carries an unfathomably large price tag, could engender its own severe environmental risks, and may or may not draw unwelcome attention from the hunters among the stars. But for AI companies, such trade-offs might still be preferable to the terrestrial roadblocks, among them an explosive populist backlash and a 12-year wait to connect to the power grid. 

Next Thursday, Google plans to launch a pilot satellite capable of processing basic AI queries into orbit, the most significant test yet in what’s become a burgeoning race to power AI from space. For now, the satellite won’t approach the levels of computing power at a hulking terrestrial data center like Project Blue. But it is a major step toward a future in which many or even most of our AI queries will be powered not from the outskirts of a city like Tucson, but from a blinking data center circling the skies.

SpaceX’s gargantuan initial public offering earlier this summer was predicated on the idea that it would launch a thus-far-unrealized fleet of orbital data centers to fuel much of the AI economy. The company — whose rockets Google will use to launch its pilot — claims that by the end of this decade, possibly as early as 2028, it will have successfully assembled an AI data center in orbit, with plans to eventually harvest a staggering 100 gigawatts of new solar energy for AI annually, equivalent to one-fifth of all US electricity consumption last year. (As always, that’s a very iffy promise.)

Still, as Google’s overtures clearly demonstrate, it’s no longer a question of if large-scale space-based data centers could be technically feasible, but rather when they’ll be ready to launch, and perhaps more importantly, whether the math of using them to power something like AI will make much financial sense anytime soon. As of now, building a one-gigawatt data center in space would cost about $51.5 billion, according to an analysis by aerospace engineer Andrew McCalip, compared with $16 billion for one on land.

“They’re not breaking any rules of physics. Yes, there are going to be engineering challenges, but they are all solvable with enough time and money,” said Jason Aspiotis, a leading space systems engineer who now leads an orbital data center startup. “The gray area is, well, what about the economics?” 

But even if this astronomically kooky plan does take off, it may not change much for those of us left on terra firma: Our energy bills will remain high, even if data centers aren’t there to drive them up. Our earthly infrastructure — affordable housing, public transit, and more — may be just as hard to build as ever. Harder even, maybe, given that crucial resources and untapped tax revenue will be literally rocketing up out of our reach.

No, if these tech giants succeed, it will not fix what’s wrong here on Earth, but merely prove once again their exemption to the gravitational forces holding the rest of us down.  

Space-based data centers, explained

My father spent most of his career working in the satellite industry, and would sometimes let me waltz inside the quaint little data center housed in his office, a room I remember as roughly the size of my school’s computer lab, hosting rows upon rows of servers that I couldn’t touch, churning with a cool, mechanical hum as they crunched zeroes and ones.

AI data centers work similarly to that server room but on an exponentially larger scale, requiring far more land and energy to run, and much more extravagant cooling systems. Sending his office’s data center into orbit would have been extraordinarily difficult, and getting one of the facilities that powers modern LLMs aloft would be an even larger, more ambitious undertaking. But the benefits — namely, “cheap space (no pun intended) and power,” as my dad put it to me recently — are even more enticing.

An orbital data center need not negotiate with an elected official like Lee over precious water rights to the Colorado River, though it would need to install bigger-than-football-field-sized heat radiators to keep itself cool. You can’t simply launch something that big into orbit, meaning a space-based hyperscale data center — the size of which would likely rival that of the International Space Station — would need to be assembled piece by piece, probably by robots. And while the sun would provide an endless supply of solar power to process your ChatGPT queries, it would also inflict incessant radiation on delicate circuitry. If something breaks, you can’t send a technician up to fix it. Many experts are also concerned that if an excess of space junk, which extraterrestrial data centers would inevitably produce, begins colliding like bumper cars in Earth’s low orbit, it could knock out the orbital infrastructure we already rely on, like GPS navigation and weather satellites. 

Even plastic decays in space, said Sven Bilén, a professor of aerospace engineering at Penn State, who, like Aspiotis, believes that early orbital data centers will probably be “processing data from space, like spatial imaging satellites or weather satellites, as opposed to, you know, cat videos” or AI-generated reality TV slop. 

But even if AI is not the lowest-hanging fruit for deploying compute into space, it is certainly the deepest-pocketed, meaning that many investors and companies like SpaceX and Google seem quite willing to shoulder the start-up costs anyway. And considering how difficult it’s gotten to build data centers on Earth, or at least in the United States, it’s indeed possible that space could become the cheaper, or at least most convenient option, more quickly than one might think. 

“In some form or another, in five years, we are going to have some orbital data centers in space for sure,” Krishna Muralidharan, who leads the Center for Semiconductor Manufacturing at the University of Arizona, told me, but it’s hard to say right now whether those consist of just a few expensive prototypes, or the beginning of a much bigger swing toward orbital compute. “Only time will tell whether you’re going to have a business case for it.”

Can we send our data center problem into space?

So, is it better to have a data center in space or in your backyard?

David Reid, a member of Virginia’s House of Delegates representing Loudoun County, home to more data centers than anywhere else on Earth, says the question has repeatedly come up at his town hall meetings. Reid is the rare politician — apart from the president, anyway — willing to defend terrestrial data centers, calling them a “money-printing machine” for local schools and infrastructure. Having spent decades working in the defense and technology sectors, he is doubtful that the math on orbital data centers will ever make sense.

“I believe this is, for lack of a better term, fictional nonsense,” he said, and is “creating a distraction from the fact that local officials here on Earth still need to make better zoning decisions.”

If most of the AI industry can eventually be powered by space, as SpaceX claims, it will require millions of sprawling new satellites, in an orbit currently dotted by fewer than 20,000 of them. Though “space is really big,” as Elon Musk has put it, one million satellites would still profoundly and, likely irrevocably, transform our night sky into one in which data centers outnumber visible stars.

“Will humanity notice if there are fewer stars in the sky? Well, not so much right now,” given light pollution in many cities, said John Barentine, an astronomer and environmental advocate for the night sky. “But what if they’re replaced by these artificial stars in the night sky? Is that the point at which people will say, ‘Wait a minute, we don’t want this. We want the benefits, but we don’t want this as a consequence.’”

This is, of course, the tension also at the heart of building data centers here on Earth. To some extent, they’re the most tangible representations of how little power we have to shape the course of a transformative new technology that few trust, and even fewer asked for. It’s one thing to object to a hulking data center, but it might be another altogether to see the sigils of our digital overlords illuminating the firmament. Would a place like Tucson, one of the most beautiful places to stargaze in the world, prefer a Project Blue to a starscape marred by blinking servers in the sky?

Update, September 24, 10:35 am ET: This post was originally published on September 3 and has been updated to include information about a pilot satellite capable of processing basic AI queries that Google is launching soon.