Amazon’s New Alexa AI Sounds Like a Dystopian Nightmare

The rollout of Amazon's new "GenAI-powered" Alexa+ blends Orwell's 1984 with the exciting world of e-commerce.

In the age of the AI boom, it seems that everything’s getting a makeover. The ill-defined software has totally revamped perfectly good products, from workout apps to creative programs like Adobe’s Photoshop to search engines like Google — unless you’re totally unplugged from the internet, the stuff is nearly unavoidable.

So it shouldn’t come as much of a surprise that the brain trust behind Amazon’s Alexa embraced it, too.

Now 11 years old, the all-seeing living room assistant is getting a fresh upgrade in the form of “Alexa+,” a fusion of Alexa’s classic data surveillance with the hallucinogenic power of generative AI.

On a fresh episode of the New York Times’ “Hard Fork” podcast, tech journalists Kevin Roose and Casey Newton discussed the dystopian changes to the Alexa platform and their experience after using the new service for a few days. Though Amazon started the Alexa+ rollout in May, it’s going extremely slowly, hitting the million-user milestone on June 23.

To start, the techies both note that they’ve been longtime users of the Alexa platform, accessed through Amazon’s Echo devices, the always-on, voice-activated gadgets which boast a history of invasive advertising and privacy concerns.

“I did not have a good experience with this thing,” says Newton, who ran the virtual assistant off the new Echo Show 5 device — a workaround to access Alexa+ instantly, versus waiting for the update to existing Echo gadgets. “Basically, what I’ve come to understand is that an Echo Show is a device that just constantly invites you to spend money with Amazon. And I found it honestly infuriating because I plugged this thing in.”

Newton said the device began simply, flashing art of his choice on its idle screen. “I would say for about four seconds per minute, it would show me some Renaissance masterpiece or something,” Newton said. “And then it would be like, hey, do you want aspirin? Do you want paper towels? Do you want to buy paper towels? You can actually buy paper towels right now. Just say, hey, Alexa, buy paper towels.”

See also  Man Follows ChatGPT's Advice and Poisons Himself

“And it was just this forever,” Newton continued, “and so I eventually just unplugged the thing because I was like, why did I just spend $90 to have a permanent rotating advertisement for household products on my desk?”

Though the duo did notice some improvements, like Alexa’s more natural voice synthesis and smoother interaction with for-profit apps like OpenTable and Uber, they noted that the update seems to have broken previously useful functions, like surfing the web, setting an alarm, and summarizing the news.

If users aren’t sure what to use their new Alexa+ for, it comes pre-baked with suggestions. Of course, these are essentially just more advertisements, as the tech reviewers discovered.

“It’s like ‘ask, me what I can do,'” said Newton. “So I asked it. And one of the things it said was, I can help you explore Gen Z music trends… so I was like, yeah, sure. Why don’t you help me explore Gen Z music trends?”

“And then it goes, well, I found some podcasts about it on Amazon Music,” he continues. “And I was like, I assumed you were either going to tell me something about Gen Z music or you were going to play Gen Z music. But now you’re trying to sell me Amazon Music, which I feel like is very consistent with how Alexa+ handles everything, which is, could we sell you a service right now? Could we sell you a product?”

Though Amazon hasn’t released much info on the “AI” behind Alexa+’s dewy superpowers — the podcast hosts speculate it’s a blend of Amazon’s proprietary AI models and Claude — it’s clear that the tech’s core features aren’t ready for the big time.

See also  Grok Is So Deeply Antisemitic That Showing It a Picture of Some Random Clouds in the Sky Prompts an Outrageously Offensive Answer

That makes it even more interesting that Alexa+ has no problem burying users in a deluge of ads. In that light, the e-commerce giant’s decision to roll the unpolished platform out to users looks more like an obvious cash grab than a quality-of-life improvement — a move typical of Amazon’s fellow tech giants.

Amazon’s new indoor-salesman approach coincides with a pretty wild rollback to Alexa’s privacy policies. Back in March, it was reported that Alexa+ would no longer abide by the old Alexa’s “Do Not Send Voice Recordings” setting, effectively allowing Amazon to store its users’ voice data on its servers indefinitely.

Whether or not Alexa+’s newfound love of commerce is worth the Orwellian privacy tradeoff is for consumers to decide. For Newton anyway, “the Echo family of devices that are just little windows that let you send money to amazon.com, they’re not for me.”

More on Amazon: Amazon Testing Humanoid Robots to Ride in Vans, Hand-Deliver Packages

The post Amazon’s New Alexa AI Sounds Like a Dystopian Nightmare appeared first on Futurism.