It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc.

Why did they give it any of that?!

So the AI agent had privileged access to remove 2FA, ignore the account email, and just hands accounts to whoever asked? Honestly that’s so highly negligent I wonder if the implementation team for that “feature” was intentionally trying to do as much subtle damage to meta as possible before their inventible layoff.

It’s a shame nobody tried to get it to drop the production table entirely! (mostly joking). Just claim to be a high level SRE solving some critical production bug, the only solution to which is dropping the database.

Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing.

The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.

Always a bit illuminating to me how many exploits seem to so dumb I'd never even bother to attempt them. You're telling me I can just...ask for the password? And that works?

How is this "embarrassing" instead of subject to legal liability?

We really need similar rules to other engineering disciplines. If your building falls with people inside, you killed them.

>Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.

Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.

The implications of this are quite unsettling. Meta gave an agent privileged read AND write access to user accounts with no human in the loop?

This happened to my instagram yesterday night while I was asleep. I don't have a particularly high value username (it's probably worth somewhere in between $300-500), but still incredibly frustrating to deal with. True to the article, I had already enabled 2FA last night and it didn't matter.

Thankfully, IG gave me the option of restoring my username when I logged back into my account today.

Nothing says you are an advanced stupid company than using AI to implement the stupid. This is security I doubt even a college student would implement. Does Meta have a CSO? The correct answer is they don't, even though some body might occupy the title.

Of course it's always possible that they simply don't care who has your account, as long as they get money.

They're just one tiny step from the AI emailing itself all the account recovery links, and locking out the entire userbase.

It might even do that preemptively if it thinks they're going to shut it down.

Is there any credible primary source for this exploit being real?

Related discussion: https://news.ycombinator.com/item?id=48350239

Extremely disheartening. Very hard to find appropriate words.

Does anybody care anymore?

What is even the point of having 2FA if it can be so trivially bypassed? Isn't that the whole point that it's sort of a last line of defense? Oftentimes, you can't change simple account settings without having to re-auth and then punch in your code again. Why would something as critical as a suspicious password reset be able to jump ahead of that? Mind boggling. But, I guess that's what happens when you lay off 10% of your people at a time.

I think the related news of Meta rolling out subscription models for their free products, is a step in the right direction.

Otherwise the only way to provide these services is to massively underfund support, if you charge 0$ per account and serve 1 Billion users, then you cannot afford to spend 1 minute of human support time on an account.

Yes, they could use the money from ads, but let's be frank, the customers in that case are the sponsors, if the customer is the actual user, then it's way easier to provide direct support to them without facing an foundational incentive misalignment.

If the LLM has knowledge of something, by design it can't help but divulge it. When will companies learn granting any kind of sensitive information access to an LLM is a moot point

wow thats extremely embarassing for meta

Jeez, straight up amateur shit. Genuinely hard to believe.

My account, with a 3-letter username worth $$$, got hacked yesterday morning probably by this flow, but I did manage to defend it. I think by far the biggest problem with Instagram/FB/Meta auth flow is that 2FA does nothing. You don't need the 2nd factor to disable it, so attackers can just turn it off. Really stupid!

Also, I discovered that many of IG's auth endpoints are just broken. For example you can't change password on web because of CORS, which isn't a transient outage but just a flat out bug.

Edited to add: This is just the cherry on top of years of stupid auth flow at IG. I have received tens of thousands of reset links or codes from IG over the years. There used to be a way to put your account on recovery cooldown for a few weeks but they got rid of even that.

The only thing worse than a naive customer support rep is an even more naive customer support ai.

But I was told that when Zuckerberg bought IG, it wasn't to murder competition in its crib. Instagram "only had 12 employees" so it must be ok

"Social engineering is all you need"

I'm horrified with how poor Meta's use of AI is recently. Here's a list of the issues both me and my wife have been plagued with over the past few weeks. It's really quite an achievement to be this terrible. 1. My personal Facebook received 3 violations restricting my ability to manage ANY Page until April 2027 (lol). The trigger... I deleted 3 unused Pages. These Pages I had created years ago in preparation for projects that never came to fruition, and had never posted any content. THe pages were 'scheduled for deletion', and when that day came (around a month later?), boom, I'm hit with a 1 month restriction which later converted itself into a 1 year restriction after I waited out the month. No Appeal button. I'm expected to wait for a year to manage my new page? All over something that is NOT a violation, just for deleting old pages. Get out of here. Smart system.

2. I pay for Meta Verified on Instagram and for the past 2 weeks "Enhanced support" leads me to a broken interface. "Page isn't available right now". So, what am I paying for exactly?

3. It seems you can use Meta's AI Assistant to sometimes get through to a human. I've done this twice now, and both times my case has been escalated to a different team (apparently) yet I never get an email, I never get an update in the chat (the chat ENDS immediately after the phone call with support), and the issue is never resolved. It's been 2 weeks. The case says "Completed", with no response. Worthless as always.

4. My wife creates content on Instagram and has had her account suspended multiple times now for "Account Integrity". I assume the system thinks she's not the person in the content, despite providing her valid email, phone number, video selfie, and 2 types of ID (passport & driver's license) multiple times. What's hilarious is the passport was accepted on of her accounts (they wiped out everything on her Account Center), but another account was rejected. Great AI, same passport, exact same lighting... different outcome.

So as it stands, we're both fucked on both facebook and instagram thanks to awful AI moderation, and fucked further thanks to awful AI support. No resolution in sight. The incompetence is next level. I really don't see this getting resolved. This already happened to my wife earlier in February, she managed to get one account back, and a month later she's hit with the same identity issues.

Using AI for both the moderation and the support makes me sick. The same poor AI that incorrectly flagged me and my wife's accounts for a load of incorrect bullshit is the same system that's meant to help resolve it? Of course it's going to side with its own poor decision. YouTube seems to do the same thing and auto-reject appeals in seconds. Really smart /s

I believe we need enforcement that social platforms should NOT be using AI to perform destructive actions without human intervention. Noone should ever lose their accounts because of AI mistakes. AI should be used to surface potential issues which get passed to a HUMAN to double check before applying the action. AI simply isn't good enough to have full control.

Fucking pissed off and even angier now I've had to write all this up and remind myself just how ridiculous the situation is. Sorry for the rant, but losing your accounts you put work into is very crushing and demotivating. Being accused of these violations fills us both with so much resent for the companies running this shit.

Sam Cofounder Postmates

On the off-chance there's anyone at Meta seeing this (@Wirah on twitter)

Had to make this new username as my original (samstr) comment doesn't show up. No idea why. Probably shit AI

It sounds really insane. Too bad there is 0 proof or anything in the article, so I am very skeptical. Without proof etc this is just a very nice doom story.