Normal view

AI could end up too cheap to control

30 July 2026 at 13:00
A humanoid robot with green eyes.
Capital markets have signaled their faith in Anthropic and OpenAI’s impending hyper-profitability, valuing each at nearly $1 trillion. | John Ricky/Anadolu via Getty Images

The AI industry’s investors and critics don’t agree on much. But many in each camp share at least one basic conviction: America’s top labs are about to make a killing. 

Capital markets have signaled their faith in Anthropic and OpenAI’s impending hyper-profitability, valuing each at nearly $1 trillion. Many of Silicon Valley’s progressive adversaries also expect the labs to grow filthy rich but fear the implications, warning that AI-induced automation could transfer vast sums of money from ordinary workers to a handful of giant tech companies. Sen. Bernie Sanders’s call for nationalizing the top AI labs rests partly on that concern. 

Key takeaways

  • The AI industry may be more competitive than investors expected.
  • Chinese labs are producing models nearly as powerful as Claude and ChatGPT — and dramatically cheaper.
  • That could make frontier AI a low-margin business.
  • A world of cheap, open-source AI would bring both promise and danger.

But recent advances in Chinese AI call all of this into question.

Over the past two months, Chinese companies have released three AI models that are nearly as powerful as America’s frontier systems — and radically less expensive. 

In June, Beijing’s Z.ai debuted a model that performed nearly as well as Claude and ChatGPT’s second-tier systems on independent benchmarks. Weeks later, another Chinese firm, Moonshot, unveiled “Kimi K3,” a model that allegedly outperforms all of its American rivals except for the very latest versions of Claude and ChatGPT. Finally, just days ago, Alibaba launched a preview of Qwen3.8 Max, which purportedly outclasses even OpenAI’s most advanced systems, while trailing only Claude’s Fable in its capabilities. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

These developments don’t merely threaten America’s AI giants with stiffer competition in the race for superintelligence. Rather, they raise a more harrowing prospect: that the AI race’s ultimate rewards will be far smaller than anticipated. In a world where new advances can regularly be leapfrogged by cheaper upstarts, hoarding the technology — and its profits — will be harder for any one company to do.

In other words, building a machine God might not be as lucrative as it’s cracked up to be. AI, it turns out, may “want to be free.”

How AI was supposed to pay off

To see how China’s new models threaten Anthropic’s profit expectations, we must first examine why those expectations have been so high.

This is not entirely self-evident. After all, AI labs aren’t much like the hyper-profitable tech giants of the 2010s. Facebook and Airbrb were relatively capital-light businesses with ultra-low marginal costs (adding a profile to Facebook or listing to Airbnb costs the companies virtually nothing). And once each gained a foothold in their respective markets, network effects enabled them to retain formidable positions without needing to constantly upgrade their products.

Building a state-of-the-art AI company is a much more involved — and astronomically more expensive — endeavor. To get to the frontier, Anthropic and OpenAI have sunk (at least) tens of billions into semiconductors, data centers, power plants, and other capital investments. Staying at the cutting-edge, meanwhile, compels them to perpetually churn out evermore costly models.

To put a new Claude model through its initial training — in which it spends months digesting the internet and sussing out statistical patterns within its text — can now cost hundreds of millions of dollars. And such foundational computation is only the beginning. A truly superlative model requires several additional months of fine-tuning. Armies of contracted experts — such as computer scientists, physicians, and mathematicians — tutor the models, grading their answers and guiding them towards better ones. Then the AI systems complete millions of rounds of practice, in which they learn through trial and error how to solve countless problems. This arduous process, known as “post-training,” compounds the costs of a single model’s development. 

All of which raises the question: Why would investors expect businesses with a cost-structure this challenging to be not merely profitable, but massively so?

There are (at least) two answers. The first (and most obvious) is that the market for superintelligent machines is liable to be vast. Frontier AI systems promise to reduce costs and improve performance in myriad white-collar sectors. And Anthropic’s soaring revenues indicate that firms do, in fact, find Claude useful. A company like AirBnB has earned billions by revolutionizing a single industry; imagine then what a technology that remade virtually all industries might be worth.

Of course, plenty of technologies are valuable but not massively profitable to produce. After all, in well-functioning markets, competition should eventually erode individual firms’ margins, even if the underlying technology continues generating huge value. 

But this is where the second answer comes in: Frontier labs’ immense costs are a burden, but they’re also a safeguard against competition — or, in industry parlance, a “moat.”

Startups may be able to afford to build or acquire more rudimentary models, many of which are “open source.” But, the thinking goes, they won’t be able to deliver Claude Fable-level performance without raising giant amounts of capital. And what investors will be willing to pour hundreds of billions into an AI pipsqueak that’s light-years behind Google, Anthropic, and OpenAI?

Alas, the Chinese AI labs’ rapid progress — and the way it was achieved — suggest that Anthropic’s moat may be shallower than previously thought.

How Moonshot swam Anthropic’s moat

The existence of powerful, Chinese AI systems is neither new nor surprising. Xi Jinping’s government has made vying for global AI dominance a key economic goal. And China’s DeepSeek, which also has stunned US companies with its lower-cost competitive models, surpassed ChatGPT as the most-downloaded free iPhone app more than a year ago.

The latest models, however, have dramatically narrowed the gap in capabilities between frontier American systems and their Chinese rivals. Just as critically, they’ve done so in a manner that other, relatively underfunded AI upstarts might be able to emulate.

Alibaba and Moonshot needed to invest massive resources to train their base models. But they allegedly found a low-cost way to refine those models into near-frontier systems: Just ask Claude.

Or, more specifically: Engage Claude in 16 million conversations, using 24,000 fake accounts. In each of those exchanges, ask the model to not only answer countless difficult questions but also, walk you through its reasoning, step by step. Then take all of this data and feed it into your own model as study material, training it to respond to the world’s most challenging queries as Claude would. 

Through this process — known as “distillation” — an AI lab can replicate virtually all of a frontier model’s capacities, without sinking vast sums into human experts and post-training computing runs. 

China’s AI labs have not admitted to using distillation. But OpenAI and Anthropic both reportedly uncovered Chinese distillation attempts earlier this year. And some of the new models appear to display tell-tale signs of distillation in conversations with ordinary users; Kimi K3 has routinely identified itself as “Claude.”

Chinese AI companies are hardly alone in using distillation to catch up with frontier labs. Earlier this year, Elon Musk admitted in court that xAI enhanced Grok’s capabilities by running distillation techniques on Claude and ChatGPT. Nonetheless, China’s latest models appear to demonstrate that distillation can help take a second-tier model to the frontier’s threshold.

America’s frontier labs have tried to defend themselves against such imitators. But this is technically difficult when distillers can assemble massive networks of bots, each asking an inconspicuous number of questions. And legally, it is difficult for America’s AI giants to argue that distillers are stealing their intellectual property. After all, in a sense, China’s copycats are merely doing to Anthropic and OpenAI what those companies did to journalists, coders, lawyers and other specialists: Feeding their public-facing outputs into a model, which then replicates their capabilities by discerning underlying patterns within the text.

Oh, and China’s giving these models away

The new Chinese models would have caused Silicon Valley enough headaches, if they merely provided stiffer competition, while demonstrating the power of distillation. 

What makes Kimi K3 and Qwen3.8 Max especially threatening to the American AI giants’ profitmaking potential, however, is that they are officially open source — meaning that the models’ parameters can be downloaded for free. (Alibaba and Moonshot have not yet released these parameters, but they say they will shortly.)

In other words, any company or hobbyist with enough computing power will soon be able to run a near-frontier Chinese model on their own hardware, modify that model to better serve a specialized purpose, and then sell access to their new version — without paying Alibiba a single yuan.

As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

For many of Anthropic and OpenAI’s potential customers, that proposition may be hard to turn down. Most businesses don’t need the world’s smartest AI, just one competent at their enterprise’s core tasks — compiling legal research, answering IT queries, writing working code, etc. A model that produces outputs 90 percent as good as Claude’s — at roughly one-sixth of the cost — will sound pretty good to many corporations.

Further, open source models aren’t just cheaper than frontier systems, but potentially more secure. If you run an AI on your firm’s own servers, then you don’t need to entrust sensitive data to Anthropic, Google, or OpenAI.

All this had led much of corporate America to embrace open-source models, even before the latest versions narrowed the capabilities gap. In a Linux Foundation survey, 63 percent of organizations reported using open-source AI systems.

And increasingly, those models are Chinese. According to Sequoia Capital, one of Silicon Valley’s premier venture capitalist firms, a majority of American AI startups now use open-source Chinese systems. As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

What’s bad for OpenAI is good (and/or catastrophic) for humanity

All this said, it is still entirely possible that OpenAI and Anthropic will justify their colossal valuations. In many highly competitive economic domains, having access to the world’s very best AI model will remain highly valuable. And America’s frontier labs still outperform all their peers. 

But it’s increasingly plausible that selling state-of-the-art AI systems will prove to be a low-margin undertaking. In a world of ubiquitous, near-frontier open source models, the AI sector’s big winners probably won’t be its top labs, but rather, its chipmakers and cloud computing providers. 

For ordinary people, a future where superintelligence is dirt cheap — and rival AI companies are constantly rising and falling, rather than consolidating into mega-corporations — would look somewhat different than the cyberpunk dystopia that the left’s been dreading. 

And not entirely in a good way. For one thing, in that reality, mitigating AI’s biggest risks would be immensely difficult. Having a handful of firms monopolize control over frontier AI systems is bad in many respects. But it does make those models easier to regulate, as the Trump administration’s decision to temporarily block Claude’s Fable in the name of cybersecurity demonstrated. 

By contrast, if recipes for ultra-powerful AI models are published all over the internet — and anyone with modest technical skills can modify them at will — then systems willing to help their users hack government bureaucracies or engineer bio-weapons are liable to proliferate.

From another angle, however, the “AI becomes almost free” scenario may look like capitalism at its finest: Retrospectively, such a development would mean that a small number of extremely rich people bankrolled the creation of an immensely useful technology, under the expectation of massive profits, only to see competition erode their returns — and disperse that tech’s benefits across a wider group of businesses and consumers. 

Granted, in the case of AI, this process might also generate a super-virus that kills us all. But hey, no system is perfect.

How public opinion is turning against AI

20 July 2026 at 14:30
Demonstrators march in a crowd while holding up anti-AI signs.
Demonstrators march during a protest against AI data centers in Vancouver, British Columbia. | Ethan Cairns/Bloomberg via Getty Images

AI was supposed to make our lives better. Instead, it’s made many of us scared and angry. Communities are protesting against the building of new data centers — the warehouses of IT equipment powering the AI buildout — across the country, and increasingly they’re winning. And polling shows most Americans think AI is moving too fast.

So how did public opinion on AI curdle so quickly? Jasmine Sun, who reports on the industry from San Francisco, argues that the backlash treats AI less as a technology and more as a political project. “The debate was not about like, is ChatGPT useful to me?” Sun told me during a taping of Vox’s The Gray Area. “The debate was actually something more like, there are these big corporations and unaccountable billionaires…coming into my city, coming into my life and changing it without having any sort of democratic input?”

Filling in for Sean Illing, I talked to Sun about the rise of “AI populism,” the parallels with the Industrial Revolution, and how the backlash could crash into the 2028 presidential election. 

As always, there’s much more in the full podcast, which drops every Monday, so listen to and follow us on Apple PodcastsSpotifyPandora, or wherever you find podcasts.

You’ve been writing about a phenomenon you call AI populism. How would you define that? What is AI populism?

I define AI populism as a worldview where AI is not seen as an ordinary technology, but specifically as an elite political project to be resisted. I came to the term while thinking about the AI backlash and the reasons people are increasingly anti-AI — whether that’s LLM slop, whether that’s Waymos in their city, whether that’s a new data center project. One thing that occurred to me was that a lot of times the debate wasn’t about whether ChatGPT is useful to me, or whether Waymos are safer than a human driver. The debate was actually something more like: There are these big corporations and unaccountable billionaires who are coming into my city, coming into my life, and changing it without any democratic input.

When I talk to people who are opposing AI in various ways, they seem more concerned with this concentration-of-power, anti-elite dimension — which is where I take the word “populism” — rather than classic AI safety concerns, which are more about the technical characteristics that might introduce risk.

You wrote a piece that touched on some of this but went to a darker place — “AI populism’s warning shots” — and you wrote about actual shots. Sam Altman, the CEO of OpenAI, was targeted by a Molotov cocktail and a shooting within the span of a couple of days. There was an Indiana councilman who voted for a data center and woke up to gunshots at his home and a note reading “no data centers.” Why do you think of those incidents of violence as warning shots of something to come?

It was pretty scary. I’m no Sam Altman fanboy, but it’s terrifying that assassination attempts are showing up in response to people’s worries about AI. One factor is that we’ve been seeing a rising wave of political violence and support for political violence in the US, especially among young people, over the past few years — the UnitedHealthcare CEO shooting, the Charlie Kirk shooting. Increasingly, a lot of disaffected, maybe nihilistic young people are turning toward political violence as a way to express political beliefs they don’t feel they have other channels for. Or maybe that person is just unwell. But I do expect to see more of it, because my theory of political discontent is that if people feel they have institutional channels to bargain for their rights — if they feel the democratic process is working, or they’re part of a union and believe their union leader will go bargain about how automation shows up in the workplace — they’ll most likely go through those channels.

When it feels like the official channels aren’t working, opposition becomes much more diffuse and volatile. That’s part of why, in creative communities, you’ll see people witch-hunting each other over AI use. I think we’ll see more political violence against people seen as AI leaders, or as supporting AI leaders.

That’s really scary.

Yeah, I’m quite worried about it. But again, my sense is that it comes from a feeling of — what else is there to be done, when you have this level of concentration of wealth and power, and there’s no democratic input right now into how AI is regulated or built?

It’s like a jump straight from complaining at your community meeting about the data center to an act of violence.

I was talking to some friends about this. During the 20th century in the US, there was a wave of factory mechanization and automation, but unions were really strong — often when a company said, “We’re going to bring in these machines,” they’d sit down with the factory union leader and say, “Okay, you can bring in the machines, but we’re going to couple that with a wage increase,” or a 35-hour workweek, or earlier retirement. There was a channel to make a deal about how automation would show up in your workplace. That meant people were more likely to accept it as something lifting all boats. I don’t think that’s happening now — most of the industries affected by AI aren’t organized in labor unions, and the democratic channels are questionable at best.

It’s like when people have agency to be part of the transition, the process goes a lot smoother. Is there a historical analogy for a technological change that didn’t allow for input from the people involved? I’m thinking of the Luddites.

The Luddites are a good example. When the automated looms were introduced, there was a lot of violence against the looms. The book I’d really recommend here is Carl Benedikt Frey’s The Technology Trap. He’s an Oxford economist who studied a ton of historical examples — in Europe, in China, all over the world — including the Luddites and 20th-century automation. His central question was: In what contexts do workers successfully stop automation, and in what contexts do they allow it to be introduced? How does the political environment, or the balance of power between people and their leaders, change the outcome? He found that when automation was introduced alongside social welfare policies — a higher minimum wage, some form of redistribution — people were much more willing to accept it, which is fairly rational.

I want to talk about Silicon Valley’s understanding of this backlash more generally. You’re painting a pretty dark picture, and you’re right in the belly of the beast in San Francisco — I’m sure you talk to people involved with AI every day. Is there a moment when it clicked for them that this backlash is real and something they have to take seriously? Or has that happened yet?

I’ve definitely noticed a huge difference, over the past six months, in how seriously people in Silicon Valley take the AI backlash.

Like what?

People just talk about it more. I’d bring up AI populism to people last year, and they’d normally say, “It doesn’t matter — technology always introduces some discontent, people get annoyed but they get used to it, like the internet.” That was the standard reaction last year. Not anymore. I think part of the reason OpenAI and Anthropic have felt pressure to introduce economic policy proposals around job automation is that they’re seeing how worried people are. The data center moratoriums and the broader data center backlash have been surprising and meaningful in getting AI leaders to recognize they have both a messaging problem and an actual problem with the product and the technology they’re introducing.

A lot of the increasing opposition to AI in Washington has caused people to see this too. At first, Trump — as you mentioned — was very pro-AI. He and David Sacks were accelerationists; they wanted AI to go faster and to block attempts at regulation.

He was the AI czar.

“The moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.”

He was the AI czar — he’s no longer the AI czar. But it turned out a lot of other constituencies, both on the left and the right, were pretty opposed. For example, Trump and David Sacks tried to introduce a big federal bill that would preempt all state-level AI regulation — no state could regulate AI for 10 years. They tried to sneak it into a big omnibus bill so no one would notice. But members of Congress realized it was happening, and — whether for kid-safety reasons or frontier-safety reasons — people said, Wait a second, the idea of preventing any state from regulating AI for ten years is crazy. A lot of people organized in Washington to successfully stop that preemption. I think that showed the scale and bipartisanship of a coalition that was very keen to make sure it stayed possible to regulate AI was underestimated. As a result of all this — the moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.

China is our big competitor in the AI race, and it certainly has all the conditions for a populist pushback to AI — youth unemployment is really high, and AI technology is in some ways more advanced at taking over real-world jobs. I was watching a video about fully automated factories and a robot pharmacist. You’re one of the rare American tech reporters who gets to spend time in China, and you wrote a piece that surprised me — you found there wasn’t really a populist backlash to AI there. Why not?

I was really interested in this question, and I was finishing my New York Times piece while in China for a few weeks, talking to both AI people and non-AI people. The main reason there’s not a big populist backlash in China is that there isn’t a lot of social unrest or populist backlash against anything — the entire MO of the Chinese government, the No. 1 priority, is domestic social stability. Any whisper of protest gets shut down; that’s why they have such strong speech controls. So one factor is that China doesn’t have much of a culture of resistance in general, whether in workplaces or politically. I’m not saying no one dissents — but it has a cultural effect too, because people don’t see it as useful or as an option. When I ask family members of mine in China about AI, sometimes they’re annoyed about specific things, but fundamentally, the idea of opposing AI is seen as almost unimaginable.

The other thing about China is that if you’re middle-aged there, you’ve lived through so many political, economic, and technological revolutions in your lifetime. When I was a little kid visiting Shanghai in the mid-2000s, there were no high-speed trains — now China has some of the best high-speed rail systems in the world. Technology has always gone hand in hand with dramatic economic advancement, with being lifted out of poverty. The modernization process has been aggressive and disruptive, but it’s not something the party has offered opportunities to resist, and it’s something most Chinese people still see as an inevitability that was mostly good for most people — because incomes did increase by dramatic amounts alongside the technological change. So I think people have a similar attitude toward AI: It’s much less about “Can I stop the AI wave?” and more “How can I take advantage of the AI wave to get ahead economically?”

We were just talking about this deep pessimism about what technology can bring us here in the US. I think a lot of people look around and think: We don’t have a cure for cancer yet, but we’ve sure seen our lives get worse in a lot of ways because of technology, social media, whatever. That pessimism probably fuels the backlash to AI, the skepticism about whether it can ever deliver on its promises. And that experience just isn’t the same in China, or probably much of the rest of the world, where technological progress has been faster and more concrete in people’s lives.

My 90-year-old grandfather said he’d love an elder-care robot to help him do tasks around the house so he doesn’t have to rely on his kids — he wants more freedom and mobility. It’s seen more as a tool to help individual goals. Even with the robot factories or pharmacies — one thing that struck me visiting a robot pharmacy was that the PR people happily said, “Yep, we’re doing these robots because human workers take too many smoke breaks and bathroom breaks and take too long.”

You’d never say that in the US, but they’re probably thinking the same thing — they just don’t say it. The other thing they mentioned is that this lets the pharmacy operate 24/7, because a lot of people need medications in the middle of the night and want to order via the DoorDash equivalent. There was actually a labor shortage before — Chinese workers weren’t willing to work night shifts — so these pharmacies are offering real consumer surplus. A significant percentage of orders come in overnight, when no other pharmacy is open. And with the factories, part of the issue is that Chinese workers, especially young people, don’t want to do factory work anymore.

“I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation.”

That anecdote gets at the promise and peril of AI, and the role of the backlash movement — which I’m still wrestling with how I feel about. On the one hand, I want to live in a world where cancer gets cured, where we live in an era of abundance, where things are cheap and easy to make because factories can run all the time with machine workers who don’t require anything — I want the future we were promised, of flying cars and everything working well.

But I also don’t want to lose my job, or see humanity wiped out by an angry machine god. Because we don’t really know what’s going to happen yet, it’s hard to work out my own feelings about the pushback here in the States — what’s appropriate, and what’s holding us back from real advances in our lives.

Totally, I agree. I like Waymos — I think they’re safer, and I’d prefer a safer robot car driv[ing] me around instead of me driving. I’m not a good driver; no one should let me drive. So I wrestle with some of the same things.

To close out the conversation — let’s come back to the United States. AI populism is brewing as a political force. We’ve seen it show up in a couple of races so far, but it’s early. We’ve got the midterms, then the presidential election. How do you think it’s going to affect American politics this November, and in 2028?

My sense is that this November, it’s going to be more about state and local races where AI really shows up. I’m going to spend some time in Michigan and Wisconsin this summer touring some of the data center sites facing the most opposition — those states also have contested governor and Senate races where AI and data centers have become a core issue, so I’m interested to learn more there. I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation — especially if we start to see some of the employment impacts people are expecting. As soon as we see something like a 2 percent rise in unemployment, if that happens, I think people will be very upset, and we should expect a ton of focus on the issue.

The other thing I’ll note is political opportunism — you’re already seeing a bit of this, where politicians are likely to raise the salience of AI above where people might ordinarily care about it, because it’s become a convenient boogeyman. It polls so poorly, people are so anti-AI and anti-data-center, AI billionaires are so unsympathetic, that no matter what your policy program is, AI is a great reason to push it. I think a lot of politicians who are being clever about this are going to move AI to the center of the conversation, raising its salience to manufacture urgency for proposals they’re already excited about. That’s definitely something I’m watching for 2028.

❌