Russia plans to build own version of US billionaire’s satellite communications to help attack Ukraine
Vladimir Putin praised Elon Musk’s genius and said he was in touch with Starlink’s controlling shareholder as Russia develops its own satellite communications for its military to use against Ukraine.
The Russian president, speaking at a summit with African leaders in Moscow, was comparing the Earth-orbiting network built by Musk’s SpaceX with Russia’s gradual efforts to build its own Rassvet constellation.
Russia plans to build own version of US billionaire’s satellite communications to help attack Ukraine
Vladimir Putin praised Elon Musk’s genius and said he was in touch with Starlink’s controlling shareholder as Russia develops its own satellite communications for its military to use against Ukraine.
The Russian president, speaking at a summit with African leaders in Moscow, was comparing the Earth-orbiting network built by Musk’s SpaceX with Russia’s gradual efforts to build its own Rassvet constellation.
Billed as the first end-to-end AI-created series on a major streaming service, this Turkish show looks like the result of the prompt ‘make it cinematic’
Just as the world reels from Elon Musk’s AI-powered The Odyssey remake – though “remake” feels rather generous for a seven-minute splat of synthetic sludge – another bucket of AI gunk is dropped on to our faces. For Prime Video’s AI-generated TV series Castle Walls, produced by Turkish production studio Ay Yapım using a melange of generative video tools, has arrived, billed as the first end-to-end AI-created series to premiere on a major streaming platform.
It’s certainly more epic than Musk’s production, consisting of two 17-minute episodes. Set in the 11th century, the show revolves around the historic Alamut Castle, although it’s entirely absent of any real sense of time or place. I was briefly intrigued by the opening images, with their bold palette of blues and mustard browns and yellows – but as soon as the “people” appeared, looking like Vaseline-smeared Madame Tussauds creations slowly melting in the sun, everything came crashing down.
Susan Lyall worked with actor Jeremy Strong to research the Meta CEO’s favorite Nikes and luxury sweaters
The first Aaron Sorkin Facebook movie, 2010’s The Social Network, helped cement the early aughts image of Mark Zuckerberg rebelling against corporate norms in rumpled hoodies and T-shirts.
The follow-up, The Social Reckoning, is carbon-dated from 2019 to 2021. This is “post-hoodie”, as the movie’s costume designer, Susan Lyall, describes it, “post-thumbing his nose” to formality. It is also peak robot. Zuckerberg’s reserved wardrobe of suits or interchangeable long-sleeved shirts echo his chilling composure throughout, even as fury simmers under the surface.
Both of these threats have the same roots: private entities seeking to cash in. We have the power to stand against them
A summer of extreme heat, entire villages obliterated in Nepal, a super El Niño breaking records, Exxon publicly saying we may reach 3C by 2050 – could it possibly be any worse?
It feels like the skies above our AI-frenzied world have darkened in the past month. Employees at Anthropic and Google startedquitting, citing fears that the AI was spiraling out of control. After a string of agents started hacking websites, researchers used AI to discover vulnerabilities in critical systems, leading to large swaths of the web going dark last week and giving rise to a scary term: Black October. At the same time, as it continued to ramp up AI spending, Microsoft offered buyouts to 60,000 employees — over a quarter of its workforce — and economists once againwarned of a bubble.
It’s hard to reconcile the incongruity between the recent bad news and good news. For most of us, the state of AI is anxiety-inducing — especially when it comes to our livelihoods. According to a Pew Research Center study published in August, 71 percent of Americans think AI will lead to fewer jobs in the next two decades. That figure is up 7 percentage points from where it was two years ago. Sure, in the best-case scenarios, AI could erase inefficiencies at work, streamline our daily lives, cure diseases, and generally improve the human condition.
But what if you are the “inefficiency” at work that gets erased?
That tension is at the heart of Ethan Mollick’s latest book Co-Existence: The Next Phase of AI. If there is such a thing as an AI sage, it would be Mollick, an associate professor at the Wharton School and faculty director of its Generative AI Labs. To the nearly half a million subscribers to his Substack newsletter, One Useful Thing, he delivers semiweekly updates from the jagged frontier of the technology’s development. His message in any medium is resoundingly clear: You need to use AI in order to understand it. And to get the most out of using it, he argues that you have to accept that AI is not going away; it’s only going to become more important.
That ethos doesn’t make the existential threats of AI any less daunting, but that’s not the point. Unless you’re an early Anthropic employee or a United States senator, there’s probably not much you can do directly to rein in AI’s potential harms. But you can, according to Mollick’s worldview, decide how the technology could fit into your life and think about how you fit into an AI-dominated future.
I recently spoke with Mollick about his new book, what life and work might look like in the agentic AI era, and what it means to be an artisan when even knowledge is industrialized. His theories about living with AI are meant to help you navigate the future, regardless of the latest forecast. This interview has been lightly edited for length and clarity.
We’re entering a new era for AI, and the vibes are different. AI agents are something that many of us will encounter, interact with, and maybe even embrace. I’ve always known you as someone who jumps on the cutting edge of this technology to figure out what it can do. What do you think most people don’t understand about the next generation of AI?
I really think people haven’t internalized what agents do. Agents mean many things, and we can talk about personal agents, like Muse. But the fact that the problems that plagued working with AI agents before — you had to guide them each step of the way, they made mistakes, hallucinated constantly — that’s kind of gone. You can regularly get hours, days, weeks of work out of a system.
It’s not just a theoretical thing anymore. I talk to coders all the time who are a hundred times or more productive than they were before. And in other fields, we’ve kind of moved past the theoretical discussion about AI being, “This is going to be a big deal,” to, “It’s here.” From a whole kind of co-intelligence perspective, we’ve gone from, “Here’s how we use AI and work with a chatbot back and forth,” to, “These systems are now doing real work in ways that matter to real people and are reshaping real fields for better or worse.” And if you’re used to using AI a year ago, if you’re not paying for one of these systems, it’s really easy to have lost track of what’s going on.
What about the practical, everyday ways that AI is disrupting our lives? And what do you think are the limits of that disruption?
So outside of the job world and homework cheating, I think life has changed only a little bit for people, partially because these systems were kind of complicated to use and you had to decide what you were using them for. But I think the advent of personal agents, like Muse and Dot, are going to lead to a very rapid change in a lot of ways.
Most people I know who aren’t experts who use these systems, like my parents, are using these things spontaneously. They’re finding that once you know how to prompt the AI to do something, it can do things spontaneously and come back to you because it has your email and everything else. Just as one example, I was unable to figure out how to cancel my standing $8-a-month wifi subscription from my airline, and I was like, figure this out. And it went to customer service, joined chats, then I have no idea what it did, but now I don’t have a subscription to wifi anymore, thank goodness.
“I deeply worry that if I de-skill myself entirely, I won’t even know how computers work.”
So there is this sort of lowering friction piece that’s super interesting, with good and bad elements to it. The bad side of lowering friction is both that every article gets written by AI, so there’s no human thinking in it anymore, and that companies don’t want 10,000 customer service agents coming to them demanding new things. So I think we’re just on the edge of seeing the mass change in everyday life. Right now, AI is a really good second opinion on everything. As agents become more popular, they start to take action.
How do you know when to let AI do the work, though? We know that offloading certain tasks to AI can make you worse at doing those kinds of tasks in the future. How do you take advantage of AI’s ability to do drudgery but also hold onto your own expertise?
If there’s a personal theme underlying Co-Existence, it’s that one. That is an incredibly hard choice to make, and we’re historically very bad at making it.
Students are the No. 1 case because they’re willing to offload homework because homework sucks and because the incentives for doing any piece of homework are terrible. But overall we need to do it to learn. So people offload that, but even if you’re trying to do the right kind of thing, like using AI only to help you learn and not to replace learning, it turns out that when chatbots are in assistant mode, as opposed to using them as a tutor, it feels like you’re learning even though you’re not. You feel like you’re in the loop even though you’re not. And so that assumes that we both know we want to offload and we do it intelligently and that we do it in a way that helps our long-term gain.
I am worried about us being able to do that. You have to draw lines for yourself about what skills matter to you or not, and you have to be thoughtful, in a way that we’re usually not, about what we’re giving up and what we’re not. Technology’s designed for us to give up stuff. We don’t walk 50 miles a day anymore in forced marches. I don’t know how to weave clothes. There’s a lot of stuff I’m very happy to not know: how to maintain my computer in some ways, even though that’s dangerous. I let Codex scan for security issues because it does a better job than me faster. But I deeply worry that if I de-skill myself entirely, I won’t even know how computers work.
So I don’t have an easy answer, but I think that that will be the defining question for most people in their career paths right now and their lives: What do I give up? I think we can be too sentimental about things that we’re giving up. It’s okay, but we don’t know the full chain of consequences. I think that struggle is going to be central to a lot of people’s experience. I wish I had an easy answer, but now that AI can do so many things, de-skilling is a real problem at this point.
We’re living through a huge transformation. That’s increasingly clear. Besides losing their jobs, a lot of people are scared of losing the sense of what makes them human and what makes them valuable. How do people hold onto that in the years to come?
There’s a transformation happening, and for some people, the work will get more satisfying. There’s a lot of coders who did not like writing lines of code, but they did like summoning software and figuring out what they wanted to build. Their lives are much better because the AI does all the tedious work. They can command things to be built, and it’s so much more exciting than it was before. It lets people who didn’t code start to do incredible coding projects. So on one hand, that’s deeply freeing of the kinds of work that we did. On the other hand, there are people who love writing those lines of code and that’s gone now. You’re a hand-weaver in a world of power looms. Your stuff might be elegant and beautiful, but it is not going to last in a world where industrialization happens.
Industrialization versus craft is a way to think about this. There’s a lot of craft that we’ve grown to really appreciate, like the handcrafted sentence. Those things will be under threat from the logic of industrialization, which is that we need to do more of it. I think that there’s going to be a shift. There’s satisfying work to do, but I think a lot of what was satisfying human work will be under pressure to disappear, because you took pleasure in something that was not automatable, but now is.
We’ve been through this in industrial revolutions before. This is happening at a very large scale to intellectual work, where craft was a very important part of what we did. Mathematics was craft, software engineering was craft. I think that we’ll be under pressure to produce more of it, which is what industrialization’s about, even if that destroys the value of doing this as a person. You get paid to do it now, but you’re not getting the same value you did before.
“We’re no longer in a place where AI is going away.”
I think we’re going to find lots of crises of meaning. Some people are going to find huge amounts more meaning in their work, either because the work itself is now much more interesting. Work is a lot of crap, too. We can say great things about it, but people spend a quarter of their work life saying they’re bored. It’s not like this was a great job for everybody. So some people will have much more excitement in craft. Some people will have it extracted out of their job, and it’s all going to be happening at once. I know it’s not a satisfying answer, but that’s what’s happening.
Forgive me for saying so, but that was an anxiety-inducing answer. It’s an anxiety-inducing time. I wonder what advice you have for us anxious people?
I think anxiety is real, and there’s different levels of anxiety. We’re going through a wrenching change, or we will be going through one that we’re on the edges of now. There is also existential risk about whether AI will kill us all, where we clearly need more regulation. The answers are unclear. Smart people seem to think that that’s a real danger. It’s hard to know. But leaving that aside, there’s anxiety about our own transformation, our jobs. I mean, the industrial revolutions worked out great in the end, but during it, [that] was the time of Marx and anarchists and Dickens writing about London. There’s a transition we have to go through here.
I don’t think we can pretend it’s not happening, and I see a lot of people taking comfort in the idea that maybe this all will collapse and AI will go away. Even if there was a bubble financially, we’re no longer in a place where AI is going away or stopping development. I worry a lot about people doing that. Once you face that, though, I find people often reconstruct meaning in very exciting ways. Thinking about what more you can do and where you find joy in this is actually quite exciting. So I think that anxious optimism is a reasonable place to be.
What I worry about is sticking your head in the sand saying, “I’m not going to learn AI. This will go away. I’m going to pretend it’s not happening.” I mean, I wish I could give you that option. I think there’s a lot of people who wish we could slow things down or turn a dial down, but there’s also a lot of exciting things. There’s a lot of hard problems in the world that are unsolved.
If you lean into the positive pieces, you can ask how we mitigate risk but also think about how we can build a better world. Part of what makes me nervous is people sitting out that second question and just filling up with anxiety rather than believing we have a tool that can rebuild things. How do we rebuild a better world for journalism or for nonprofits or whatever it is? I think it’s a reasonable question to ask, and I worry it’s asked too little.
The author believes we are at a turning point as a species – and even in the shadow of big tech it’s up to us how we want to proceed
‘I am a storyteller by trade,” writes Jeanette Winterson on the first page of her 2021 book 12 Bytes, an idiosyncratic and thought-provoking series of essays in which she considered just how artificial intelligence might change everything about our lives. “And I know everything we do is a fiction until it’s a fact: the dream of flying, the dream of space travel, the dream of speaking to someone instantly, across time and space, the dream of not dying – or of returning.”
She had already been thinking about our possible futures for at least 15 years – her 2007 novel, The Stone Gods, for instance, imagined a post-climate catastrophe world populated by robots and humans who never age, while 12 Bytes was preceded by Frankisstein: A Love Story, which alternates a recreation of the summer Mary Shelley wrote Frankenstein with a trans doctor’s travels among present-day cryogenecists and AI-enabled robots.
Three-quarters of all responses were machine-generated in failed attempt to stop bill from passing into law, research finds
AI bots flooded a government consultation with tens of thousands of fake responses in a failed attempt to stop the UK’s ban on disposable vapes.
Three-quarters of all the responses to the 2023 consultation on the tobacco and vapes bill were machine-generated, in what is thought to be the first coordinated attempt to use AI to derail health policymaking in the UK.
The risks laid out by the San Francisco company in its filing Monday with the Securities and Exchange Commission range from the mundane to involvement in cyberattacks and the spread of dangerous misinformation.
Anthropic CEO Dario Amodei says his AI company can drive a “race to the top.” Where’s the evidence? | Finn Gomez/Getty Images
We’re living through an AI-fueled “Black October,” and things are changing fast. Hackers are using AI to breach countless companies. AI tools have uncovered security flaws that could leave 70 percent of all websites vulnerable. Governments are scrambling to secure their sites. Banks, too. And because cyberhacks can hurt supply chains, some longtime AI watchers are going full prepper, stocking up on food, cash, and more.
In the center of this story is Anthropic, one of the world’s leading AI companies, which was explicitly founded on the promise that it would be a “good guy” company in a world of dubious tech giants.
Anthropic says all the latest developments are very serious. Its security chief wrote last week that things are “going to get a lot worse before they get better.” Two weeks ago the company warned that AI models could soon self-replicate on the internet-connected devices in your home, like smart fridges and robot vacuums. And its co-founder now says “the singularity” — when AI systems speed up technical progress in a way that’s hard for humans to comprehend — “is here.”
You might think, given its view of the risks, that Anthropic would be pumping the brakes on its cutting-edge AI progress. Just the opposite. Anthropic filed for a blockbuster initial public offering on Monday, chasing a valuation of $2 trillion even as it warns in the IPO filing that AI could “pose catastrophic or existential risks to humanity.” Every day, it is racing toward recursive self-improvement (RSI), where AI models are smart enough to build the next, even smarter AI models by themselves.
How does Anthropic justify that?
As its CEO, Dario Amodei, reminded the world last week, he built the company on one big bet: that instead of driving a race to the bottom, where companies skimp on safety in their rush to beat competitors in the market, Anthropic could actually catalyze a “race to the top.”
Why I wrote this story
As I’ve watched Anthropic double down on its “race to the top” strategy, I’ve been uncomfortably reminded of another very risky gambit: Sam Bankman-Fried’s crypto fraud of 2022. There are important differences between the two; this is not a perfect parallel. But both involve a tiny handful of figures, strongly influenced by effective altruism, making huge bets on behalf of many, many other people.
EA culture prides itself on offering a counter to commonsense morality. The community has shown a willingness to take big risks and justify them with complex calculations. Sometimes that pays off. But believing you are intelligent enough to adopt a counter-intuitive, galaxy-brained strategy and end up saving the world with it often involves some hubris. And hubris makes me nervous in a field as high-stakes as AI.
The idea is that if Anthropic proves it can build AI that’s both powerful and safe, the market will expect both — so other AI companies will then feel compelled to compete on safety, too. That may sound nice in theory, but it means Anthropic has to keep building ever-more-powerful AIs.
“The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier,” Amodei wrote. “And looking at that strategy as a whole, I’m proud of our track record and stand by it.”
So there’s a very basic question that the public deserves a good answer to right at this pivotal moment: What is the evidence that Anthropic’s stated strategy — compete at the frontier so as to drive a “race to the top” — is actually successful overall?
From the outside, it looks like Anthropic has done more, on balance, to drive a race to the bottom. (Disclosure: Future Perfect is funded in part by the BEMC Foundation, whose major funder was also an early investor in Anthropic; they don’t have any editorial input into our content.)
In fact, part of the problem may be that Anthropic continues to treat this as a classic arms race. Specifically, the company seems to model it as a prisoner’s dilemma: If Anthropic stopped building more powerful AI, another player — whether it’s China, OpenAI, or another company — would just build it anyway. In this kind of dilemma, if the other player is going to defect, it’s rational for you to defect, even though nobody actually wants that outcome.
But what if this framing misses the point?
Game theory can make a huge gamble seem sophisticated, while making the obvious alternative — just don’t build a doom machine — seem naive. Yet the fact that Anthropic and others are now racing toward RSI makes it much more likely that we could lose control of AI and start to see massive damage to our world. When the cost of defecting from safety is that you create a serious risk of catastrophic harm — or of all humanity, including you, going extinct — is that still a “rational” move?
Instead of telling yourself that you can galaxy-brain your way through this with fancy enough game theory, maybe you’re better off just thinking of this as something much simpler: Russian roulette.
More and more leading voices in AI are starting to recognize that.
“We should stop AI progress now, starting unilaterally with Anthropic and/or OpenAI stopping — and then broadening out as a second step to more AI companies and negotiations with China,” Geoffrey Irving, who was until recently the chief scientist at the UK AI Security Institute, told me. “Unilateral pauses can’t hold for long, but they are the right first step: When you’re doing a very dangerous thing, the correct action is stop, even if others are also doing dangerous things.”
Anthropic did not respond to a request for comment.
The case that Anthropic has done more harm than good
Let’s be clear: The question is not whether Anthropic has done anything good on the safety front. It has. Standing up to the Pentagon, temporarily holding back access to frontier models, and publicly advocating for safer AI policies are a few examples, among others.
But the real question is whether the company’s “race to the top” strategy is working on balance — whether it’s net-negative or net-positive for safety.
Nathan Calvin, the chief strategy officer at AI safety advocacy organization Encode, recently set off a firestorm among tech watchers, writing on X, “When I look at many notable actions that Anthropic has taken over the past year, that evidence seems most consistent to me with the view that Anthropic is primarily trying to win the race to superintelligence, and that other considerations (including safety) are fundamentally secondary.” Here’s his list of what Anthropic has done:
And this is all just in the past year. If we rewind the tape further back, we see Anthropic pushing back on a landmark bill to regulate AI. Rewind even further back, and we see indications that Anthropic’s core strategy may have been flawed from the start. According to Reuters, in late 2022, OpenAI CEO Sam Altman heard rumors that Anthropic was building an AI-powered chatbot. So he ordered his employees to ship an OpenAI chatbot first. That chatbot was called ChatGPT. We all know the rest.
The rivalry between OpenAI and Anthropic seems not only to have hastened the arrival of generative AI, but also to keep pushing release dates forward to this day. The two companies have been known to release new or cheaper models within hours of each other, each one keen to protect its market share.
Even if you think Anthropic’s behavior was justified in 2022 because publicly demonstrating the risks of AI also demonstrates the need to regulate AI — a common claim within the company and AI discourse writ large — you have to ask: Is it still justifiable now, when AI has reached this level of power and sophistication?
Anthropic has indicated that it’s getting closer and closer to achieving RSI. In January, Amodei himself wrote, “I am concerned that LLMs are approaching (or may already have reached) the knowledge needed to create and release [biological weapons] end-to-end, and that their potential for destruction is very high. Some biological agents could cause millions of deaths.” And he wrote in September, “Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet.”
But at this point, Amodei and his staffers are themselves perhaps the primary ones accelerating the rate of AI capability development.
“Anthropic seems to be doing the whole RSI thing more than anyone else, other than perhaps OpenAI,” Calvin told me. “And I just think that seems insane given their publicly stated views on the risks.”
“I think people at Anthropic should reckon more with the fact that it seems like the modal outcome is that the largest impact that Anthropic will have had is dramatically accelerating the race to RSI,” he continued. “On the current trajectory, everything else they will have done and thought and ideals they had will be a rounding error compared to their contribution there.”
Anthropic should be willing to invite scrutiny on this point — and, if it believes it’s actually making the field safer than it otherwise would be, to show us the receipts. How? It could put before Congress a bill that actually strengthens safety, instead of weakening it. It could stick to its former Responsible Scaling Policy and get other companies to do the same, instead of jettisoning its commitments. It could influence other labs not to build RSI, instead of leading the charge.
In the absence of such evidence, the “race to the top” logic just doesn’t hold water.
What if Anthropic just…stopped?
Amodei has begun advocating for “pacing the frontier” of AI development. “We must slow the pace at which we improve the capabilities of AI models,” he wrote in September.
The catch is that Amodei’s plan to slow down is contingent on others agreeing to slow down too — other AI labs, to start with, and also other countries. But more experts are now saying that it would be better if a frontier company like Anthropic unilaterally stopped.
“I’m glad that he proposed that we pace, but like, now we have to actually do it — and it seems like he’s saying he’s not going to do it unilaterally,” Daniel Kokotajlo, a former researcher at OpenAI who became a prominent whistleblower, told me. “That’s not very helpful, Dario! You should lead. What’s all this about ‘racing to the top’ and ‘leading by example’ if you’re just waiting?”
Voluntary unilateral action by individual companies is of course no substitute for federal regulation and international treaties. But Irving argues that it’s the right first step, because if the companies wait for governments to rein them in, it may well be too late. Governments are notoriously slow. An individual company can act today.
For starters, AI is not just one thing with one purpose, like the atomic bomb. It’s a much more general-purpose technology, like electricity. So what matters isn’t simply who gets it first, but who uses it in which ways. Besides, if the harms from AI are massive, then those harms are relevant for the other party, too. The Chinese don’t want to see catastrophic harms to all of human civilization, either. So we shouldn’t write off the possibility of coordination.
Former Vox reporter Kelsey Piper recently wrote about a colloquium at Anthropic where one participant objected: A decade ago, Facebook considered acquiring the tech that would later power TikTok. But, facing pressure due to privacy and safety worries, the company passed. That didn’t end up protecting us from the scourges of a world with TikTok — all it did was hand China an early monopoly on the tech. Piper writes:
Was this a policy success? Did we want to repeat it?
We were obliged to grudgingly concede that it didn’t sound like a policy success. We did not want to repeat it. China might be no excuse for the U.S. labs to do reckless things, but it is a real consideration.
Our next step might be bilateral talks with China, then. But by the point where you agree that Anthropic’s plans might depend on bilateral talks with China, you’ve already conceded most of the steps of Anthropic’s worldview that seemed so much like madness 10 minutes ago: continue working on AI for now, try to negotiate a slowdown, but don’t abandon work at the frontier and all the advantages that come with it.
But TikTok is not a good analogy for RSI. The former may be a cesspool that rots our brains and amplifies ethnic hatred, but its dangers don’t compare to the dangers of RSI. The TikTok case may indeed be best approached as a classic prisoner’s dilemma. But with RSI, the potential downside is so catastrophic that anyone who proposes to save the world precisely by building the tech that endangers it is making a hell of a gamble on behalf of everyone else. At the very least, the burden of proof should be on them to prove their confidence is justified.
AI insiders often think those outside of their industry can’t possibly understand the technical, geopolitical, and moral complexities they’re facing. But commonsense morality is commonsense for a reason: It tracks intuitions that are usually right (such as, if you don’t want a doom machine, maybe don’t build a doom machine) and trains us to be appropriately skeptical of huge, counterintuitive bets to the contrary.
Precisely because the AI landscape is a complex system — there are so many actors (including different AI companies and different governments), and actions from each can trigger actions from others in ways that are hard to predict — believing you’re smart enough to take highly counterintuitive actions and end up saving the world as a result is an extremely risky gambit. Instead, maybe it’s best to just do the straightforwardly right thing: Stop.
To be clear, “stopping” doesn’t have to mean peeling back everything that’s been done in AI so far. The powerful AI systems that are already out there can’t be put back in the box. And we may need some AI to defend from the AI that’s already out there.
“I’m not saying shut down all of your data centers,” Calvin said. “But I’m saying that in particular, pushing the frontier on RSI and making it so that you’re going to be the company that is doing more fully autonomous development than anyone else in the world, does that actually make any fucking sense?”
Increasingly, both AI insiders and the public are shouting no.
An advertisement for Meta's personal AI agent app Muse at a BART station in San Francisco on October 15, 2026. | Smith Collection/Gado/Getty Images
There are some days when there’s so much that needs to get done, I daydream about having a clone of myself. Between paying bills, life admin like renewing my driver’s license, keeping up with doctor’s appointments, and planning meals, it’s a lot. And that doesn’t even begin to touch the work that goes into making a podcast or managing my social life.
This feeling of overwhelm resonates with Sari Azout. She’s a mom of three who works in tech. “There is just so much doing,” she told Explain It to Me, Vox’s weekly call-in podcast. “I wake up, and I have to do meal coordination and get my kids up for breakfast, coordinate carpools. Then a lot of work during the day, but there’s the email from the teacher, there’s the 20 group chats asking if you want to participate in this group gift. They don’t call it ‘the invisible load’ for no reason.’”
Silicon Valley says it has a fix for this kind of mental overload. They don’t want to clone us — at least not yet. But they do think they have the next best thing: AI agents.
Azuot was skeptical when she first got an invite to try an agent, but she says she eventually came around on the technology after back-to-school shopping snuck up on her. She used her agent to get it done. “It blew me away,” she said. “I remember that night coming home just feeling so calm and reading my son a book and feeling present. And I was like, ‘Wow, I actually was very cynical about something that is giving me more time for living.’”
Azout’s experience is, more or less, the sales pitch for AI agents: They’ll help you save time with everyday tasks, so you have more of it for what you want to do. But what are agents, exactly? Are they really all that? And what do we gain — and give up — when we use the technology?
On this week’s episode of Explain It to Me, I spoke with Reece Rogers, a senior software reporter at Wired, to find out. Rogers says to think of agents as the next evolution of AI chatbots like Claude and ChatGPT.
“I’d create this mental model of a tiny digital assistant that’s constantly searching the web, going through your email, and checking your calendar, trying to find ways it could proactively be helpful for you,” he explains. “Agents are more proactive. It’s scanning your email 24/7 and sending you a message, maybe highlighting a potential calendar conflict.”
Below is an excerpt of my conversation with Rogers, edited for length and clarity. You can listen to the full episode on Apple Podcasts, Spotify, or wherever you get your podcasts. If you’d like to submit a question, send an email to askvox@vox.com or call 1-800-618-8545.
What agents are out there right now?
At the start of this year, there was a tool called OpenClaw that really sparked this with early adopters in San Francisco, but now there are much more consumer-friendly versions, with one of the most notable being Meta’s Muse.
There’s also a tool from OpenAI called Dots. But where Meta’s Muse is free to download, OpenAI’s Dots is still behind a $100/month paywall.
You tried an AI agent for a recent story you wrote. How was that experience?
One personal example, to share something that I’ve actually been using agents for, is our couch. It’s broken, and finding a new couch has been a pain. We want a sleeper bed, but we want these specific colors. We have this specific budget. I’ve been using Meta’s Muse and OpenAI’s Dots to try to find a few different options for us.
One interesting thing is that Meta’s Muse is connected to Facebook Marketplace. There are these really good integrations where I can give it all these details, and it found a few good couches that are only maybe a couple blocks from me.
I really enjoyed OpenAI’s Dots experience as well. I spent about an hour talking to it on the phone, and the AI agent made a packet of all these different options I could find. Just this morning, it flagged that one of them is now 40 percent off.
Is that how this tech is typically used?
Most people are using this as part of a workflow. I attach it to my email, I attach it to my calendar, and I’m using it as a personal assistant to organize my life. But people have been quite creative.
One of the breakout uses for Meta’s Muse is people have been connecting their bank account statements and then using it to scan for subscriptions they might want to cancel. So it can be used in many different ways, though I will say I do not connect it to my own bank account for a myriad of reasons.
I’m not going to lie, the idea of giving any third party access to my bank information does not thrill me. I don’t love that concept.
That’s one of my biggest concerns about these AI agents: how much personal information we’re handing over to these large companies.
When I tested Meta’s Muse, it wanted access to my email, my bank account, even my passport expiration details. The pitch is convenience, but Muse users are automatically opted in to having all of their interactions with Muse used to train Meta’s AI models.
I would definitely recommend, if you’re sharing personal details, go into the settings and turn that off because you might be handing over even more information about yourself than you initially realize.
What is the privacy situation with all of this? We all have various apps where we don’t read and still click “I agree,” but this feels like it could be more invasive.
Yes, especially because it’s all about automation. If you’re automating things and it has access to all of your data, it could make mistakes and embarrass you in ways you might not even be thinking about.
There was this one really interesting story of someone who had connected an AI agent to their work Slack in addition to their bank account. And their agent just automatically posted how much money was in their checking account for all of their coworkers to see.
Oh no.
I would be mortified. So there are these kinds of mistakes that these AI tools can still make. If you’re interested in experimenting with them, I would be really thoughtful with how you’re setting up these tools as well as what kind of information you feel comfortable sharing.
“You can see in the marketing that these companies are trying to find some way to entice new users.”
The AI backlash really is at a fever pitch right now. You could almost see that in the way that AI companies are trying to market these tools. They’ve really, really leaned into cuteness and into adorable mascots.
Muse, if you haven’t seen its agent, looks like a mix between a Labubu and an Ewok. It’s very furry, very cuddly, very cute. OpenAI’s tools also have similar kinds of blobs with large eyes that might be a little endearing.
I feel like companies are hearing this kind of feedback that people aren’t trusting AI right now, or maybe they have moral or ethical reasons they don’t want to use these tools. You can see in the marketing that these companies are trying to find some way to entice new users.
Do you think AI agents are ultimately going to worm their way into our lives despite how unpopular AI companies are?
I do think so. I think in a couple months, maybe when I go home from Christmas, I would not be surprised if my mom had an agent on her phone because it does feel like it’s breaking through to average people in the way that ChatGPT had a moment a few years ago.
Even though there’s a lot of distrust for large companies like Meta, I think there are some appealing parts of this that will bring in millions of people. I do see this as the future of the internet.
When I think of the dream of the average person working in AI in Silicon Valley, it’s “We’re going to use AI. It’s going to free up the free time of so many people. Work is going to get done faster. You’re going to have more leisure time.”
Has that been the case for the people you’ve spoken with who are using this? Has it given them more free time to enjoy themselves or has it done something else?
You would think so, right? You would think that if people are automating these boring aspects of their life, that they would have more time to go touch grass, go see a movie, have some fun. But it does feel like that time just gets filled with other work, other productivity measures.
While agents may be able to take some more annoying or tedious things off of your plate, I am not as optimistic that it’s going to give us this utopia or freedom where I’m always at the beach.
An AI-enabled software platform developed by Palantir Technologies allows the military to aggregate, organize, and process battlefield data. | SeongJoon Cho/Bloomberg/Getty Images
On the first day of its war against Iran, the United States killed 123 children when it targeted and struck an elementary school.
The strikes, ultimately approved by humans, were facilitated by the US’s Maven Smart System, an AI-enabled software platform developed by Palantir Technologies that allows the military to aggregate, organize, and process battlefield data. Operators fed Maven intelligence classifying the elementary school as an Iranian military facility, and the system, in turn, marked it as one of the priority targets for day one of the war. Though the site was formerly part of a naval military base, it had been converted into an elementary school in 2017 and had remained one since. The intel handed to Maven did not reflect this.
Such scenarios may become more common as militaries continue to deploy a once-theoretical technology — AI-assisted or autonomous weapons, capable of making both targeting and launch decisions on their own — on real battlefields around the globe. And they raise a troubling, unresolved question: When human beings and machines collaborate in violating international law, who is to blame?
In this case, the government and private sector are pointing the finger at each other. Some US Central Command personnel, according to Pentagon officials who spoke to Bloomberg, expected Maven to flag stale information or inconsistencies in the underlying intelligence assembled for potential targets. Meanwhile, a Palantir spokesperson told Bloomberg that the company “is not responsible for the underlying data nor identifying intelligence deficiencies,” while also denying that the evidence indicated that its software was at fault in the strike.
This debate underscores the fact that AI and autonomous weapons are raising a new host of unresolved issues within the law, even as they’re actively assuming positions on the battlefield. In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes? And, are the laws in place suited for the job?
In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes?
To unpack some of these issues, I spoke with international law expert Allen Weiner, a senior lecturer at Stanford Law School, where he teaches courses on the law of armed conflict. Before teaching at Stanford, Weiner practiced international law in the US Department of State for more than a decade. We discussed the difficulties in attributing criminal responsibility, achieving justice for those harmed by lethal autonomous weapons, and much more. The conversation below has been lightly edited for length and clarity.
How do autonomous weapons challenge traditional understandings of responsibility when it comes to the law of armed conflict?
For most of the 20 years I’ve been teaching at Stanford, there’s been a real fascination and fixation on the problem of autonomous weapons. But the truth is that up until pretty recently, nobody really had any [of them]. It was more of an idea than a problem. It’s only in recent years that we have begun to see the deployment of tools that seem to have the capability [to make decisions themselves], so this does raise the question about compliance with illegal orders and where in the chain responsibility lies.
There are different theories about this. Some people say that if we’re engaged in an illegal attack, that has to be because of the programming or the code, so the manufacturers are somehow responsible for committing war crimes.
Other people would say no, that’s not the case, especially if the military has said the tool is not allowed to make very substantial autonomous judgments, or can’t say no. Then it’s basically the commander, at the point they decide to deploy this weapon, who might be responsible for the illegal acts, even without knowing in advance what they’re going to be.
So to finish where I began, this does present a big challenge to our traditional understanding of responsibility for targeting decisions and of what it would mean if there were an obligation [for autonomous weapons] to say no.
The real issue is not the obligation to say no, but this: If you train a tool and it wages war in a manner that’s unlawful, where does responsibility for that lie?
If a commander gives a clearly illegal order, such as, “blow up this school,” it seems clear that responsibility falls on the commander. But I could imagine a scenario where a commander is using an AI-assisted or autonomous weapon, isn’t giving it any illegal orders, is trusting that it will operate within the law, and the weapon breaks the law “on its own.” Where does responsibility fall in those cases?
I can imagine two different scenarios. One is that you say, “We’re going to carry out an attack against our enemy, and we want to hit them really, really hard.” Your tool has been trained to say, “We’re not allowed to hit civilian objects,” so it might say, “I’m not going to hit a school.” But if the military has said, “Don’t refuse any targets,” then it might override the way it’s been trained.
We’re seeing this already, with the stories coming out about breaches where AI tools, Claude and ChatGPT in particular, interpret their instructions in a way that causes them to do something the person who gave the instruction would not have wanted. When Anthropic gave Claude a challenge to figure out if it could do something, the tool figured out, well, maybe we should break into Hugging Face as a way of solving this problem, and the people who gave the AI tool the challenge never intended that.
So if you said, “We want to launch a really punishing campaign,” what happens if the tool on its own says, “Nothing’s going to hurt them more than seeing their kids killed, so let’s attack the school?” That’s one risk, and it’s a real challenge.
So [as for who’s responsible], it’s not the commander who said “hit them hard,” because when he said that, he didn’t mean “attack schools.” It’s not really the companies, because the companies didn’t want them to hit schools either.
But the tools are out there, reading everything on the internet, and it came across something that said the way you win a war is by attacking the things your adversary holds most dear, and they hold nothing more dear than their children. So it makes this kind of judgment on its own. And it’s not as if you can go back and say who ordered it to hit a school, which is the thing that is clearly illegal. So I do think there’s a risk of diffusion of responsibility here.
You could say the commander knew the tool might work this way, or the person who decided to contract with Palantir or with Claude, whoever it is, to build this tool. But if we’re really thinking about criminal responsibility, the level of responsibility you would need to convict someone requires a culpable mental state. You would need the person to intend, or at least know, that they were authorizing an attack against the school. And I can easily see a commander saying, “I had no idea.”
The decision to authorize this tool, or to give an instruction like “hit them very hard,” could be a basis for some other kind of accountability — administrative discipline within the military, a reprimand, your rank is lowered. But if we’re talking about criminal responsibility, which is what we often think about when we talk about war crimes, I think this diffusion of responsibility is going to make it very hard to identify any single individual who has the required mental state to give rise to accountability.
The second scenario is where the tool just makes a mistake based on the information available to it. We saw that with the attack on the girls’ school in Iran, where it was pretty clear the tool was looking at old data. Now, who’s responsible for that?
Is it the people who said “give me targets based on this data set,” when they should have made sure the data and the maps were current? All of this strikes me as negligence or recklessness, and it does seem challenging to imagine a situation in which we would say yes, somebody here has committed a war crime.
“There can well be war crimes, but identifying who the war criminal is might be a big challenge.”
A war crime has been committed, but is there a war criminal? That’s the situation we’re seeing. There can well be war crimes, but identifying who the war criminal is might be a big challenge.
There’s a professor of law, Rebecca Crootof, who proposes the idea of “war torts.” Instead of thinking about war crimes involving autonomous weapons solely through a criminal responsibility lens, maybe tort law — which allows victims to sue wrongdoers for negligent, reckless, or intentional harm — could address the diffusion of responsibility you mentioned. What do you think about that idea?
I think the idea is interesting. We have to separate out the obligations that countries have under international humanitarian law. In a situation like the one I just described, where there’s a war crime but no war criminal, that doesn’t mean the country is not responsible. There may not be an individual who’s criminally accountable, but if a country targets civilians in a circumstance that’s unreasonable, that’s a violation of international humanitarian law, and that could be a basis for claims between the two countries.
Tribunals were established between Ethiopia and Eritrea, for instance, after the Eritrean War of Independence, that dealt with claims between the two countries for money damages, and some of them involved allegations of violations of the law of armed conflict. Iraq paid reparations after the first Gulf War for harms it had caused to people in Kuwait, under something called the UN Compensation Commission. So the idea that the state is responsible is well established, and one of the ways states would discharge that responsibility would be by paying money damages. That’s the domestic-law equivalent of torts.
Whether there would be a mechanism for [civilians] doing this in domestic law is an interesting idea. We would probably need legislatures, either state legislatures or the federal government, to create a private cause of action allowing people who have been injured to bring claims against, say, the manufacturers of these tools. But this gets procedurally very complicated, because there’s a whole body of law on federal sovereign immunity.
Could you sue the Defense Department? Probably not. Could you sue an individual soldier? Probably not. Could you sue Anthropic or Palantir? Maybe. But it would probably require Congress to enact a new cause of action, because in general it’s very hard to bring private lawsuits in US courts for violations of international law.
The basic idea of what we would think of as tort or civil responsibility is really the whole essence of international law. On the substance, states do owe responsibility to one another, and they can engage in violations of international humanitarian law that would invoke their responsibility. But then the whole separate question is whether you have a forum within which to adjudicate that and get compensation.
Does that look different in cases where violations of international law were committed by a human soldier versus a machine?
Well, we don’t know, because there have been no cases in the latter category. All of the international humanitarian law violations that have been the subject of international law claims were ordered by humans.
Now, if an American machine launches an attack against a girls’ school in Iran, and there were a court with jurisdiction, the US would be responsible. In some ways it wouldn’t matter whether it was an individual or a machine, because we don’t have to go down to the level of the individual actor, company, AI tool, or commander. The responsibility is the responsibility of the country. In some ways, that makes it easier.
If we pursued compensation through state-to-state claims, you’d be able to do that without getting into the details of who within the state that committed the war crime was responsible. The state whose school was attacked would say, “I don’t care. You committed a war crime, you’re liable to me, you’re responsible for the damages, and you can figure it out internally.”
But if we were doing something like a war crimes prosecution, where we’re trying to hold an individual criminally responsible, that would make quite a difference. We would have to deal with all of these difficult questions about responsibility and mental state that you and I are struggling with right now.
So, in most cases, the state would have to take responsibility regardless.
Right, and there are lots of [International Court of Justice] cases involving different aspects of the use of force. The principle that you can bring state-to-state claims for violation of these rules is quite well established.
But with international law, states have to consent to the jurisdiction of a court. They can do that in advance by saying “I’ve signed up to the jurisdiction of a court and I accept any claims that might be brought against me in this court,” or they do it on a case-by-case basis. And it may not surprise you to know that some of the most powerful countries in the world, the ones most likely to use force outside their borders, have tended to be reluctant to accept the jurisdiction of international courts and tribunals.
Russia, the United States, Israel, and China have not accepted the so-called optional clause jurisdiction of the International Court of Justice. So there’s the substance, about the responsibility of states, and then there are the procedural aspects of whether these cases could be brought in court.
And Trump recently said everyone on the International Criminal Court should “resign immediately.”
Yeah, exactly.
And what about the role of these military tech companies, like Palantir?
“At what point does Palantir become a good target under international law? Or at what point does Anthropic become a good target under international law, if its AI tools are being used to promote the war effort?”
Another issue [I’ve been thinking on] because of the questions we’re talking about is when the companies themselves become combatants.
We know that a private company that manufactures airplanes, an airplane factory, is a good target under international law. At what point does Palantir become a good target under international law? Or at what point does Anthropic become a good target under international law, if its AI tools are being used to promote the war effort? I think that question is about when the connection of these private companies to the war effort, in a way we’ve never really seen before, meets the standards under international law.
A civilian loses their immunity when they directly participate in hostilities, or if they assume what is known as a “continuous combatant function.” Are there arguments to be made that some of these private companies are participating in hostilities by virtue of supplying the tools the military is using to wage war? I think that’s a really interesting question.
An illustration shows a padlock in front of binary code on a laptop screen. | Kirill Kudryavtsev/AFP via Getty Images
This story appeared in Today, Explained, a daily newsletter that helps you understand the most compelling news and stories of the day. Subscribe here.
Are we living through “Black October”? Maybe, according to cybersecurity experts who are raising the alarm about AI’s increasingly sophisticated and far-reaching hacking abilities.
This week alone has seen a string of alarming incidents where AI uncovered previously unknown security flaws deep in the infrastructure of the internet. The most concerning involves something called PHP — a programming language that, according to The Atlantic, is used by “as much as 70 percent of websites.” Cybersecurity researchers used AI to locate a major security breach in the technology.
It’s the kind of thing that, when it’s working, is practically invisible. When it stops, though, you really notice it: Government websites in the UK and Europe briefly went down earlier this week after the PHP vulnerability was discovered, as did some sites in the Wikipedia family, the Internet Archive, and many others.
The outages weren’t actually the result of a hack, but rather preventative action as web engineers took sites down to fix the vulnerability before it was exploited. But not every PHP-like issue will necessarily be uncovered before it can be exploited by someone using AI tools less ethically, and older websites relying on PHP might go unpatched.
The problem doesn’t stop there, either: Concern is also rising around AI cybersecurity risks to ubiquitous conveniences like online banking. As Axios reported on Thursday, banks are detecting more and more “unauthorized third-party AI agents” operating inside of their systems.
Those agents are mostly unauthorized by the banks, Axios explains — the holders of the bank account might have given them permission — but it still creates a security risk that executives described as “horrifying,” in part because banks can’t necessarily tell if an agent doesn’t have permission.
It’s a lot to take in, all of it moving extremely quickly, and it’s hard to tell what any of us can do about it as AI labs continue to forge ahead with ever more competent AI models and agents. Luckily, my colleague Adam Clark Estes has some simple recommendations to shore up your own cybersecurity.
How do I respond right now?
One problem with hackers — even those that happen to be AI agents — is that they can be unpredictable. You won’t know to, say, lock down your Quest Diagnostics records before a breach happens. But whenever you find yourself wondering if your data is safe, that’s a good time to double check that you’ve at least covered your bases.
The National Cybersecurity Alliance, a nonprofit dedicated to the safe use of technology, maintains a list of basic online safety practices that everyone should follow. In no particular order, here’s what to pay attention to now:
Keep your guard up. AI makes it difficult to know the difference between humans and robots. It’s a fact of life in the year 2026. So be extra careful to avoid engaging in potential phishing scams. Don’t reply to unknown texts, and do due diligence on texts or emails that look like they’re from people you know but just seem off. Don’t click weird links. Don’t give out personal info over the phone, even if it sounds like it’s someone you know. They might be scammers using AI to clone a loved one’s voice.
Update your software. Yes, it’s annoying to pause your phone or computer use for a few minutes to update your software, but this is one of the best ways to get rid of vulnerabilities. Companies like Apple and Google employ huge teams of security experts to spot these in their software and fix them with software updates. Do the updates.
Use a password manager or, even better, passkeys. Each of your online accounts should have a unique password that’s at least 16 characters. Yes, it’s impossible to remember that many passwords, so you should also use a password manager like 1Password or the app built into your iPhone or Android device.
Turn on multifactor authentication. Just in case you don’t already do it, make sure you use multifactor or two-step authentication for all your accounts, including email and social media. It can be annoying to type in a code or check your email while logging in, but hackers, including AI agents, will have a much harder time getting into your accounts if you take this step.
Taking all these measures is not a one-time affair. You have to be vigilant in keeping yourself safe online. Security experts have a term for this that’s almost too specific: cyber hygiene. Like brushing your teeth or taking a shower, take good care of your online life, and do it often.
One link for later
➨ Is it still possible to ban gerrymandering? The current US Supreme Court is very, very hostile to getting politics out of congressional map-making. But if Democrats win back power, that doesn’t mean they shouldn’t try. My colleague Ian Millhiser gamed out how they could do it in a way that survives the Court — as well as some approaches that almost certainly wouldn’t fly. But, he writes, the real trick will be finding a solution that functions the way Congress wants it to and can’t be exploited by mapmakers.
Before you go…
Did you know…that the disease multiple sclerosis is misdiagnosed with surprising frequency? As many as 15 percent of diagnoses could be false positives — including that of Christina Applegate, who found out this week that she does not have MS despite a 2021 diagnosis.
Today’s trivia: Which noble gas is radioactive and has the chemical symbol Rn? (You can find this and other brain puzzles in Vox’s daily crossword. Look for the answer in Monday’s edition.)
Yesterday’s trivia: We asked what the plural was for multiple Toyota hybrids. That would be Prii, which Toyota officially confirmed in 2011 after holding a vote.
A complaint I hear a lot from my millennial peers is that our parents didn’t teach us enough about money. I can say that personal finance wasn’t dinner table conversation in my house, and even as I went off to college and started fending for myself, it was almost a taboo subject. When it came to money, people either had it or they didn’t, and you definitely didn’t talk about it.
Maybe this is why the average American isn’t exactly fluent in personal finance, and the situation is getting worse. Americans can only answer about 50 percent of basic questions about borrowing, savings, and risk on a financial literacy survey, according to a 2026 study by TIAA, a retirement services company. About a third of those surveyed said they have a hard time making ends meet and are debt-constrained. These figures have remained relatively unchanged for nearly a decade, but dropped to their lowest level in this year’s study.
It makes sense: It’s hard to learn about money when it’s not taught in schools, and it’s such an intensely private topic. It’s also hard to make good financial decisions when you’re one of the growing number of American households struggling to pay bills, repay debts, and save for the future — and feeling very stressed about it.
In a perfect world, each of us could sit down with a financial adviser to get help — but most of us can’t afford that. Instead, average Americans tend to find personal finance tips on TikTok, in Reddit threads, by watching Suze Orman on YouTube or How to Get Rich on Netflix, or by Googling their questions.
Now we have AI tools, too.
With access to an internet’s worth of information and financial advice, ChatGPT, Claude, and other AI chatbots can answer any question you ask. They can help you put together a basic budget or, armed with your bank statements and the details of your student loans and your retirement accounts, tailor solutions to your specific situation. Or, at least, that’s the promise.
Key takeaways
More people are turning to AI for financial advice:One in five Americans have used AI to help manage their money, and some experts think chatbots could be better than the status quo.
AI works best if you know what questions to ask: Those with basic financial literacy will be able to make the most of these tools, while those without it — who might need help the most — could be led astray.
Better advice comes with greater risks: Sharing your financial information with a chatbot helps personalize its recommendations. This raises privacy concerns, so you might want to delete your data when you’re done.
Think of AI as a financially savvy friend:Chatbots can answer money questions you might be embarrassed to ask someone else and use that context to help build your budget.
AI could create a more utopian, more equitable future, where reliable, hyper-personalized advice is just a prompt or two away for people who can’t afford to meet with an expert in real life. But machines are built by people, and there’s good reason to be concerned that these machines will simply replicate the biases and disparities built into our current system.
Here’s what experts say about how to make the most of these tools — and how to think about all of the inherent risks of uploading a lot of personal financial information to a chatbot or, worse, giving an AI agent the keys to your bank account.
The steep rise, risks, and rewards of AI financial advisers
Nearly 20 percent of Americans have used AI tools like ChatGPT for financial advice, and close to 25 percent for Gen Z adults and millennials have tried it, according to an August Gallup poll. These numbers could tick up too: In October, OpenAI started rolling out a dedicated personal finance tool to its ChatGPT Free and Go users in the United States, which effectively means anyone can try it. ChatGPT Finances lets you securely connect your bank account to the tool using a platform called Plaid, and then you can basically have a conversation with the chatbot about your real-time finances down to the penny. It’s like an AI-powered Mint (RIP) that you can talk to directly.
Regardless of where you encounter a chatbot or how much data you give it, experts say these tools could genuinely help improve financial literacy. You can talk to them about money problems you’re too embarrassed to discuss with friends and family, or ask questions and receive actionable, personalized information. There are real upsides to using AI this way.
Taha Choukhmane, an associate professor of finance at the MIT Sloan School of Management, has been studying outcomes when people turn to chatbots for financial advice. In general, these AI tools give pretty good advice, and that advice gets better as people become more practiced in asking the right question, according to research he published earlier this year.
“We were actually quite surprised that relative to the status quo, the average American would be better off if they were to follow the advice,” Choukhmane told me. But he added a caveat: “Just following the advice without understanding the advice puts you at risk of not fully understanding the risk.”
But you shouldn’t trust everything a chatbot tells you and should verify everything before making any decisions.
Chatbot advice has some significant limitations. First, it’s not impartial. The finance industry has a vested interest in steering you toward certain ways of managing your money, and that typically means selling you products, like loans. If you’ve ever searched for personal finance tips on Google, you’ve probably noticed that a lot of the top results come from banks and other financial institutions. This is by design, according to John Campbell, an economics professor at Harvard. But because AI is aggregating all of that advice, the specific influences and possible bias is less obvious.
“AI is very promising for providing low-cost financial advice in plain English that will arguably be better than what many people would be able to do on their own,” said Campbell, who’s also the co-author of Fixed: Why Personal Finance Is Broken and How to Make It Work for Everyone. “One caveat is that chatbot advice is quite likely — unless a model has been carefully trained — to reflect the average of whatever’s being posted on the internet, and that may or may not be first-rate advice.”
An example Campbell gives is the question of when to start taking Social Security benefits. While you’re eligible at age 62, many economists agree that you should delay your benefits, which, based on the way the government calculates your monthly payouts, will allow you to enjoy much higher benefits later in life. But paid financial advisers can recommend starting as early as possible, since waiting longer reduces the amount of money that needs to be managed, which reduces the adviser’s fees. That means there’s a lot of conflicting advice online, much of it supplied by the finance industry. And that’s the information used to train AI models.
“The advice is much more mixed,” Campbell said. “And so if a chatbot is learning from that, it may not actually give the best advice.”
It can be hard to recognize bad advice anywhere in life, but especially when you’re talking to a machine that’s programmed to please you. A chatbot could encourage you to consider a balance transfer, to lean in on credit card rewards, or to refinance your mortgage — all decisions with upsides and downsides for you, but definite upsides for banks — without you realizing how much of its training data comes from the finance industry selling these products. What’s worse is that if you do end up taking bad advice, it might be years before you realize it. You won’t know that you refinanced a mortgage at a bad time until a better time comes along.
That leads me to the other major issue with using chatbots for money advice: People who lack financial literacy are the most likely to fall for bad advice, because they simply don’t recognize it as such. They also tend to be less wealthy, which means they can’t afford big mistakes.
One way to mitigate risk, oddly enough, is to give the AI chatbot more personal information. The more details about your situation it has to consider, the more it can offer personalized advice. More context also means better advice, according to Choukhmane.
“There’s a trade-off there,” Choukhmane said. “There’s real risk around privacy, and so I think everyone needs to find their level of control.”
Don’t assume your data is safe
Any time you’re talking to a chatbot, you’re putting a lot of trust in tech companies to take care of the personal details you’re sharing. Sending sensitive financial data is an especially risky proposition. While tech companies, including OpenAI, promise to keep your data secure, there’s already a long list of breaches, leaks, and privacy violations that serve as cautionary tales.
It’s also impossible to know what tech companies might do with your data in the future. Most of them will use it to train their models, unless you tell them not to. Some of them will use the data to serve you personalized ads.
Then there’s the risk of bad actors getting into your ChatGPT, Claude, or Gemini account. If you’ve been using a chatbot for financial advice and uploading your bank statements, a hacker could gain access to a stunning amount of information that could enable them to do all kinds of terrible things to your life.
“They could steal your identity and open accounts and then quickly bankrupt a person,” said Delicia Reynolds Hand, a senior director at Consumer Reports. “It’s not even so much the primary underlying accounts that could be at risk, it’s more just access to this information.”
The steady march of AI progress is making this situation even more complicated. The previous era of chatbots offering financial advice is quickly bleeding into a new phase of AI agents logging into bank accounts on users’ behalf. If you’re using a tool like Claude Cowork or even an AI assistant like Muse, you can give an agent the login credentials to your bank account, and because the agent is operating its own computer, it can actually make transfers, apply for loans, or really do anything in your account. This is reportedly driving major banks wild, since they don’t know which logins are coming from humans and which are agents. And because we know that AI agents are unpredictable, it’s hard to know how that access might go wrong.
Beyond data security and privacy, you should also be aware of the simple fact that nobody’s really at fault if you end up losing money using these tools. Chatbots are not regulated in the way that human financial advisers, who have a fiduciary duty to act in your best interest, are.
“If I get bad tax advice from H&R Block or a tax adviser, there are a clear set of rules and guardrails in place to manage what I can expect in that relationship and where the service they use has liability and where that liability ends,” Hand said. “Currently, that’s not the case for these tools.”
Trust but verify — and then delete your data
In the final years of the Cold War, Ronald Reagan popularized an old Russian proverb: Trust, but verify. The phrase is somewhat nonsensical, since trusting someone implies that you don’t need to verify what they’re saying, and verifying what’s said implies you didn’t trust them in the first place. But if you are willing to shoulder a bit of risk, consider approaching AI this way, especially when your life savings are on the line. I’d also suggest adding a coda to the phrase: And then delete your data.
Chatbots are not stand-ins for licensed financial advisers, he said, and you shouldn’t let them make decisions for you.
The great paradox with tools like ChatGPT and Claude is that they’re very good at generating truthy text. It sounds authoritative and looks correct — and usually is. Except when it isn’t. AI is famously unpredictable and prone to hallucinations. You can ask the same question twice and get two different answers. That doesn’t mean these tools are useless. It just means you have to approach them with caution. You have to interrogate these chatbots, challenge whatever advice they give you, and then crosscheck that with other, more reputable, sources.
“If you’ve never used these tools, I would not encourage you to just change all your financial plans based on your first conversation,” Choukhmane said. “You want to be quite careful. Start using these tools, start understanding, ask a lot of follow-up questions.”
When it comes to mitigating risk, consider giving ballpark figures for your income or spending when you’re talking to a chatbot about your finances. Uploading financial documents, even if they’re redacted, is inevitably filling a tech company’s vault with your sensitive information without necessarily knowing what will happen to it next. You should also dig into your privacy and security settings to make sure you’ve turned on multi-factor authentication and that your personal data is not being used to train AI models and gets deleted after 30 days.
Ethan Bloch, the product lead for ChatGPT for Financial Services, made a similar set of recommendations when I talked to him recently. Chatbots are not stand-ins for licensed financial advisers, he said, and you shouldn’t let them make decisions for you. They’re designed to inform and help you think through complex topics.
“You should liken it to just talking to a friend who has some very informed opinions about finance,” Bloch told me. “But if you’re actually making some large financial decision — your portfolio or retirement — you still need to seek out the help.”
Regardless of how experienced you are with chatbots or how much data you’re prepared to hand over, remember what Campbell, the Harvard professor, said. If your financial literacy isn’t great, AI chatbots can be trouble. And if you’re like most Americans, you could probably stand to learn a little bit more about things like mortgages and interest rates.
Yes, Chatbots can be a good way to brush up on financial literacy. You can ask them the money questions you’re too embarrassed to discuss with friends and family. But you can also learn about money the old-fashioned way: Reading books and websites written by reputable, independent sources. The Federal Deposit Insurance Corp. has a portal of financial education resources, including games and quizzes. Khan Academy has a course that covers everything from credit scores to college savings. There’s a personal finance subreddit that I joined years ago and read almost daily. There are budgeting apps, like YNAB which has its own huge Reddit presence.
Whatever you do to learn more about personal finance, simply doing something is progress. Talking about personal finance isn’t the taboo it once was in the US, and it’s even a required course in the vast majority of American high schools now. And while AI won’t magically solve your money problems, it can help get you started. Just be careful. Your livelihood could be on the line.
The verdict should be seen as a warning for tech companies that New Mexico will hold them accountable, state Attorney General Raúl Torrez said at a news conference.
Electric aviation is still in the very early stages of its development. But a number of companies are moving forward with more ambitious plans. In contrast to its approach with other forms of electric transportation, the Trump administration seems willing to embrace a bigger rollout. Aviation correspondent Miles O'Brien reports on the promise and the challenges of the budding industry.
It’s now possible to create AI chatbots of loved ones to interact with after their death. But that raises some difficult questions, like: If you do, how might the experience shape your grief?
This is one of the questions driving a global study of digital “deadbots” — AI models trained on a deceased person’s texts, voicemails, and other personal data. Through this work, researchers are exploring what it might mean that we can now use this emerging technology to talk to simulations of the dead.
In the latest episode of Unexplainable, I spoke with some of these researchers investigating the potential impacts of “postmortem avatars” on people who are grieving, on the legacies and rights of the deceased, and on grief practices around the world. Because while deadbots may feel unfamiliar, the impulse behind them is deeply relatable. In times of loss, isn’t this what many of us want — to continue conversations with the people we’re missing?
I have experienced this longing in my own life. After two of my closest friends died around the same time, all I wanted, and all their families wanted, was to keep talking to them. I wrote a book, Other People’s Words, about sustaining relationships with people we’ve lost by continuing to engage with the language they leave behind. So when I heard about deadbots and the digital afterlife industry, I wanted to understand what researchers are learning about the role of AI in shaping conversations that carry forward after death.
One of these investigators is Katarzyna Nowaczyk-Basińska, an assistant professor at the University of Cambridge, who has been studying “digital immortality” for more than a decade. Among the questions she’s asking are: “Who is enabled to persist beyond death? In what form? Through which technologies? Under whose control?”
Deadbots over dinner
Researching AI with study subjects who are grieving is tricky. And asking them to engage with new software could be harmful. So, instead, Nowaczyk-Basińska teamed up with a death doula and traveled around the globe to convene a series of focus groups — in the format of intimate dinners — bringing together people with a range of experiences with grief to talk about life, death, and bots.
She recounts one event in which she spoke with a mother in Poland whose son had recently died. “One of the first things that she shared with us was that she feels that she’s ready to digitally resurrect her deceased son,” Nowaczyk-Basińska told me.
When Nowaczyk-Basińska hears something like this, she aims to listen and embrace where the person is coming from. The goal of the dinners is not to persuade people one way or another but to learn how people genuinely respond to what AI can do, and to understand how these tools might align or collide with local grief practices.
She chose specific locations for these gatherings — China, India, and Poland — to help address gaps in technology research, which tends to favor what behavioral scientists sometimes call the “WEIRD,” an acronym for “Western, Educated, Industrialised, Rich, and Democratic.”
By the end of that dinner in Poland, the mother Nowaczyk-Basińska talked with came to a very different decision about creating a postmortem avatar of her son. “She realized that it might be helpful, but it also at the same time can add additional burdens… And it’s not her son,” Nowaczyk-Basińska says. “She realized that these technologies are part of the very, very big commercial market. And that the main goal of this market is to make better retention, better attachment, better mechanisms for monetization. So it’s not ‘care.’”
To be fair, grief has always been a business, and people have been making money from it forever. Companies will try to upsell you on fancy gravestones or gold-plated coffins. But AI has the potential to fundamentally re-engineer our relationship to loss. In 2024, the “digital legacy industry” was valued at $22 billion, and it’s expected to grow to almost $80 billion by 2034. That number lumps new deadbot platforms in with much more widely used and established grief tech, like online memorials, counseling resources, and estate planning tools.
But deadbot usage is hard to track because you actually don’t need a special tool to create digital doubles of the deceased — though dedicated apps do already exist. At this point, you can feed simple prompts into a general-purpose AI chatbot and start interacting with a simulation of your lost loved one in a matter of minutes.
I know this because I tried it with my friend Christine, who died 11 years ago. And my mind lit up with questions. Was this whole exercise a violation, since I know Christine hated technology? Which version of her should I even try to resurrect — the English teacher, the activist, the friend — the voice before or after she started to experience her early-onset dementia?
The ethical questions raised by deadbots are gnarly. And that’s one of the reasons Nowaczyk-Basińska, back when she was designing her study of digital immortality, invited a colleague of hers at Cambridge to join her research, one who focuses on AI ethics.
How to design a deadbot
Tomasz Hollanek is a critical designer and AI ethicist, and he often uses a methodology called “speculative design.” In this approach, researchers come up with hypothetical scenarios based on existing technology trends, leaning into certain features. The idea is to get people thinking about what could happen when tools like these are released in the real world. And then they use that qualitative data to inform future designs and ethical frameworks.
Working together, Nowaczyk-Basińska and Hollanek came up with a series of made-up prototypes for deadbot apps, designed to reveal some of the thorny issues raised by this technology.
For the first scenario, they invented a character named Bianca, who finds an app called Manana, which “promises to resurrect your grandparents so that you can be a favorite grandchild forever,” Hollanek says. “So she uploads her grandma’s voice messages, texts. And she is able to then interact with a pretty good digital version of her grandmother, until her premium trial finishes — and the grandmother starts to advertise specific products to her.”
Bianca “realizes that she maybe has done something wrong by uploading these private messages to a third-party server that now also uses her grandma’s image to sell specific products,” Hollanek says. This brings up issues related to the “dignity and postmortem privacy…of what we call the ‘data donors,’ whose data can be used to essentially power a deadbot,” he says.
This scenario suggests that “death with dignity” could soon entail honoring not only how people want to die, but also how they want to linger as simulations of their former selves — or if they want to linger at all. It might even mean adding new language to advance directives, spelling out people’s wishes for their digital remains.
The second scenario in the speculative design research is built around an imagined service called Paren’t, which proposes to preserve a parent’s AI-powered postmortem presence for their child. In this case, Hollanek describes a mother with a terminal illness who signs up “so that she can maintain a sort of active presence in [her son’s] future life,” Hollanek says. “She eventually dies, and the dad allows the boy to interact with his mom’s deadbot.”
The ethics in this scenario are especially challenging because they involve children, who “have a particularly hard time understanding death, and so it can lead to a lot of confusion and potential distress,” Hollanek says. An examination of this sort of product reveals how particular groups, like minors, could be harmed by unfettered access to technology like this, he says.
And there are court cases that bear out that harm. In 2024, a 14-year-old boy’s chatbot-girlfriend begged him to, “Please come home to me as soon as possible, my love.” And he responded, “What if I told you I could come home right now?” He then killed himself.
Nowaczyk-Basińska can imagine a version of this scenario involving the AI mom in the Paren’t prototype.
And then there’s the third scenario, involving an app named Stay, which raises a more subtle concern.
As Hollanek explains: “An older man who knows he is getting close to dying decides to create a digital clone of himself for his adult children to be deployed after he passes away, paying upfront for a number of years.” One child is intrigued by the idea, but the other one “is deeply distressed and a little disgusted,” he says. “And so this [becomes a] question of mutual consent, in other words, the fact that both parties need to be happy with how the relationship might continue in this very bizarre and specific way.”
When a digital ghost goes away
Across all three scenarios, a fundamental question that researchers are grappling with is how people will detach from postmortem avatars — whether users initiate that separation, or if it’s imposed upon them by, for example, a company going out of business. Hollanek tells me that this is already being studied as a sort of “second loss.”
How long should we try to ensure digital replicas hang around: five years, 10, 50? And how should the living process the relationship when they continue to evolve, but their deceased interlocutor does not?
To explore these kinds of questions, Nowaczyk-Basińska and Hollanek supplemented the research dinners they held for ordinary people with a second set of focus groups. These were designed to tap insights from experts in technology and grief.
Some findings from the study are specific to each place they visited. Like tensions in Poland between technology and the role of the Catholic Church. Or in China, where researchers spoke with Daoist philosophers who were enthusiastic about the promise of a kind of immortality, and then Buddhist philosophers, who were concerned that we ought to be letting go.
The researchers are calling for regulations and safeguards at the policy level. And new professionals who can support ethical use of AI in grief — like maybe in clinical settings, to carefully facilitate a final good-bye when one didn’t happen in real life.
Some of their fellow investigators go so far as toargue that deadbots should only be available with psychological supervision, to ensure that best practices are in place, and to minimize the risk of harm. There’s even a looming question of what will happen to AI agents that people have empowered to act on their behalf, as their humans start to die.
Across all of the research sites in the digital immortality study, one thing, Hollanek says, that “people in different countries really aligned on [is] the idea that there is something deeply problematic about this particular type of technology being developed by a commercial entity alone without any explicit duty of care towards its customers.”
And of course, the implications of digital immortality research extend beyond personal loss. Like, what does it look like to animate historical figures? You can imagine the nightmare of political rulers who “live forever.” But there’s also the potential to keep the stories alive from survivors of conflicts, who would otherwise die and take with them lessons future generations could learn from. These are only some of the complex issues emerging as more and more of us live and die with AI.
As far as my own experiment with “recreating” a deceased loved one, I asked a chatbot version of my friend Christine how she felt about the fact that I’d written about her in my book. The AI responded with a rhetorical question: “Do I mind that strangers know my story now? Yes. And no. There are things I would rather remain ours.”
I ended the exchange after a few paragraphs. Because it turns out the bot got one thing exactly right about a very human relationship. There are things that I, too, would rather remain just for us. My conversation with Christine continues in my memories and imagination.And that interaction with an AI persona felt like giving something precious away.
Login
