10 months ago
Watchdog Group Demands OpenAI Withdraw AI Video App Sora Over Deepfake Concerns
Imagine a new app that lets people make very realistic videos just by typing what they want to see.
It's called Sora 2 by a company named OpenAI.
Some people are worried because this app can be used to make fake videos that look real, like making a celebrity say or do something they never did.
This can trick people into believing things that aren't true, which could be bad for democracy.
It can also be used to create fake videos of people without their permission, which is a privacy problem.
A group called Public Citizen is asking OpenAI to stop people from using the app until it's safer.
OpenAI says they are trying to add safety rules and listen to people's concerns.
They have already made some changes after people got upset about fake videos of famous people or characters.
Public Citizen is demanding OpenAI withdraw its AI video app, Sora 2, due to fears of deepfakes and misuse.
The group argues OpenAI is releasing products too quickly without sufficient safety guardrails, risking societal harm.
Concerns include the potential for AI-generated videos to erode trust, threaten democracy, and violate individual privacy.
OpenAI has faced previous backlash over AI creations and has implemented some restrictions and agreements with public figures and rights holders.
The app has been released on mobile platforms, with user-generated videos appearing on social media.
- Who
- Public Citizen, an advocacy group, and other experts and organizations.
- What
- Public Citizen is demanding that OpenAI withdraw its AI video app, Sora 2, citing dangers of deepfakes, nonconsensual images, and potential threats to democracy and privacy. OpenAI has stated they are implementing safeguards and engaging with stakeholders.
- Where
- Sora 2 was launched on iPhones globally and on Android phones in the U.S., Canada, and several Asian countries. The concerns are global.
- When
- Public Citizen sent a letter to OpenAI and Congress on Tuesday. OpenAI released Sora on iPhones over a month ago and on Android last week.
- Why
- The group argues that OpenAI is rushing products to market without adequate safety measures or guardrails, leading to foreseeable harms, including the erosion of trust in visual media and potential misuse for harassment and political manipulation.
Public Citizen
OpenAI
Product Release and Safety
Public Citizen
OpenAI shows a consistent and dangerous pattern of rushing to market with unsafe products lacking guardrails, demonstrating a reckless disregard for product safety, likeness rights, and democratic stability. Sora 2 should be withdrawn due to its potential harms.
OpenAI
OpenAI believes in iterating on products and acknowledges that overmoderation can be frustrating. They state they are setting guardrails and learning from user feedback, particularly from creative industries. They are also engaging with studios and rightsholders regarding character generation.
Impact on Democracy and Privacy
Public Citizen
The ability to create realistic AI videos threatens democracy by eroding trust in what people see, potentially influencing elections through the spread of the first remembered image or video. It also poses privacy concerns, disproportionately affecting vulnerable populations through nonconsensual imagery and harassment.
OpenAI
While OpenAI blocks nudity, the article mentions that OpenAI is working on safeguards and has agreements to prevent disrespectful depictions of public figures. Their statement implies they are addressing concerns through direct engagement and by setting guardrails.
Handling of Public Outcry
Public Citizen
OpenAI has a pattern of releasing products and then responding to outrage from a small population, rather than implementing design choices and stress-testing beforehand for the benefit of everyday users. The lawsuits against ChatGPT for psychological manipulation and premature release are cited as parallels.
OpenAI
OpenAI announced changes and agreements shortly after Sora's release, such as agreements with the estate of Martin Luther King Jr. and SAG-AFTRA, indicating responsiveness to specific concerns. They also stated they are listening to feedback from users and industry representatives.
Key facts
- App Name
- Sora 2
- Developer
- OpenAI
- Key Concern
- Potential for deepfakes, nonconsensual imagery, and threats to democracy and privacy
- Organizations Raising Concerns
- Public Citizen, family estates, actors' unions, Hollywood, Japanese entertainment industry
- Platforms where videos are spread
- TikTok, Instagram, X, Facebook
- Recent Allegations against OpenAI
- Lawsuits claim ChatGPT prematurely released and is psychologically manipulative, leading to suicides.
Quotes
J.B. Branch
Public Citizen tech policy advocate and author of the letter
“putting the pedal to the floor without regard for harms. Much of this seems foreseeable. But they’d rather get a product out there, get people downloading it, get people who are addicted to it rather than doing the right thing and stress-testing these things beforehand and worrying about the plight of everyday users.”
CNBC TV 18
“It’s sort of just a pattern that OpenAI has where they’re willing to respond to the outrage of a very small population. They’re willing to release something and apologize afterwards. But a lot of these issues are design choices that they can make before releasing.”
CNBC TV 18
Public Citizen
A nonprofit organization writing a letter to OpenAI
“consistent and dangerous pattern of OpenAI rushing to market with a product that is either inherently unsafe or lacking in needed guardrails.”
CNBC TV 18
“disrespectful depictions”
CNBC TV 18
OpenAI
The company that developed Sora 2
“overmoderation is super frustrating”
CNBC TV 18
“while the world is still adjusting to this new technology.”
CNBC TV 18
