Science & Tech · AI · 1 hr ago

More than 400 everyday-looking apps hid abusive deepfake tools

More than 400 everyday-looking apps hid abusive deepfake tools

An investigation found more than 400 connected apps and domains that appeared to offer ordinary services, such as pet portraits or workout coaching.

The network’s apps were available in Apple’s App Store and Google Play, and appeared to be run by Chinese developers and a Hong Kong-registered company.

Under certain conditions, an app could switch from its advertised service to a hidden tool that made explicit images and videos from photos users uploaded.

Some templates depicted sexual assault or threats with a knife, raising the risk that people could be humiliated or threatened with abusive images.

The apps were promoted through thousands of video ads on Facebook and Instagram, and users could pay to create deepfakes.

Apple removed 19 apps after receiving details of the investigation, while Meta said it acted against more than 10,000 related ad accounts.

Apple said developers changed content after approval, and it had recently strengthened protections against this tactic; the investigation does not say what further action Google will take.

3 sources

Related

What people are saying

Paul BouchaudLead researcher at AI Forensics
This is an organized, commercially run operation: One operator behind hundreds of disposable apps and domains, cloaked to pass App Store review and pushed through tens of thousands of Meta ads under fake payer names.
via Wired
They don’t expect enforcement to catch up, and so far it hasn’t.
via Wired
AppleCompany statement
We review every app before it’s published, and these apps complied with our rules when we reviewed them.
via 9to5Mac
After approval, their developers changed the website the app pulls its content from, replacing what we approved with pornographic content, and promoted it through pornographic ads hosted on a third-party platform.
via 9to5Mac