Same thing happens with Apple Intelligence. You might join a waitlist for dinner reservations and get a text that says you'll be notified when your table is available. And then the summary will say something like "Your table is available"!
I'm the kind of person who is posting on HN about AI - I know this stuff isn't perfect and take AI summaries with the appropriate grains of salt. But I have to imagine it's insanely confusing/frustrating for a pretty sizable fraction of people.
> As an on-call engineer, this is the point when you start questioning your life choices. You know that the issue is affecting thousands of users, but the offending phrase doesn’t appear anywhere in EQL’s codebase, aside from some very old launches several years ago.
Dark days indeed.
Aside from the AI angle, there’s actually another way this weird bug can manifest.
If you’re following best practices and sending plaintext alternatives with your HTML email, then some mail clients will use the plaintext for the summary snippet and render the HTML when you open the email. So if a developer copies the success templates to the failure templates but only updates the HTML and forgets to update the plaintext alternative, then you will see this exact behaviour. It’s also pretty tricky to catch when manually testing because not all mail clients act this way.
I don't know how these things are deployed, but I imagine they are using sub billion parameter count models?
It doesn't make sense to use high parameter count ones at least due to costs.
But then I feel there is a disconnect in adopting AI. We are accustomed to chatgpt, claude, etc being really good at following instructions and summarize content, but in reality those are too expensive to host, so we end up with really dumb ai being integrated everywhere.
Maybe I'm wrong here? I know a fair bit about the landscape of local ai models for personal use, but I'm not sure how this is done when you need to summarize a billion emails a day.
This is because a VP knows that if they roll the feature back, they have to admit to everyone that they were overzealous and take the PR hit. Besides, it works great 99% of the time and most people don’t really care.
Consumer UX with LLM’s is proving to be way harder than a lot of people thought.
Tangential but I’m really over this artificial scarcity thing. I’d be happy to give my money to Nike/Jordan brand for a pair of those sneakers, but they decide to make tiny quantities to keep the hype up. I guess it works well for them but it sucks for customers, just like too many things these days.
In the mid-2010s Crawlera was being used to send all requests through a new IP, people were buying gift cards to avoid a Cybersource check, and Shopify's api was being abused. Just to buy sneakers.
Since then it has gone to another level and sneakers are just a category within the greater bot world. The "cook groups" that came with school shutdowns and the rise of Discord/Telegram are often run by talented young devs who started out learning lua for Roblox and then moved on to C++/Rust/Go.
It's hard enough to keep up and I do not envy OP. Having to deal with Yahoo's incompetence is the icing on the cake.
Sometimes I feel like I’m going crazy with this stuff. Like, what were they thinking? How did they ever imagine, even for a moment, that this would be okay? Or, even passing that hurdle, that it would be something that anyone would want?
Ah, one more reason to access e-mail through Thunderbird then...
The worst is they're effectively legitimizing spam/phishing. The simplest one is:
"Your Netflix account is on hold, and you need to update your payment details to avoid closure. NOTE: Update your payment details with Netflix. here"
Umm: FROM: buildingcounter@crgov.com
...and as the article states: no indication that it's an AI summary, and all "technical" details (eg: the email from address, the url it links to) are suppressed by default.
[dead]
That article was confusing af for me.. It's about "sneakers" meaning the type of shoe!???
> Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated
This seems like one of the greater sins here. Why in the world would you ever replace the actual subject that people have been expecting to see in that location for older than I've been alive?