The rapid advancement and widespread adoption of deepfake technology marks a pivotal moment in the development of the technological landscape, introducing a higher degree of uncertainty to the online ecosystem and putting individuals, institutions, and businesses that fail to adapt at an increasingly greater risk. Read on to understand the far-reaching implications behind AI deepfakes and learn how LegitScript can help you uncover risky deepfake offerings among your merchants or sellers.
AI Deepfakes Are a Convincing Part of the Online Information Landscape
Deceptive videos and images created using generative artificial intelligence (AI) are rapidly becoming a more common and convincing part of the online information landscape. The potential ramifications extend beyond individual privacy, with disruptive implications in the arenas of commerce, society, and national security.
The term “deepfake" refers to a piece of media, such as a video, audio clip, or image, that has been manipulated and/or synthesized to misrepresent someone as doing or saying something they did not.
AI deepfakes leverage advanced machine learning techniques, particularly deep neural networks, to create highly sophisticated and realistic manipulations of media that require little investment of time and energy. With billions of dollars of venture capital flowing into image-generating software powered by artificial intelligence, it has never been cheaper or easier to create convincing photos and videos of things that never happened.
Developments in Deepfake Technology Have Far-reaching Implications
These developments in deepfake technology have far-reaching implications for manipulating public perception and eroding trust in institutions. Deepfakes can fabricate convincing videos of public figures and media personalities making false statements, potentially influencing public discourse, damaging reputations, and challenging our understanding of reality. Businesses and brands are no more immune to the risks posed by AI deepfakes than individuals or public institutions.
The Potential for Economic Sabotage Is Substantial
With the fallout of fabricated media having a real impact on operations and public image, AI deepfakes have also added a new dimension to concerns surrounding personal privacy. The ability to superimpose someone's likeness and/or voice onto explicit or compromising scenarios poses a significant threat to individuals' personal and professional lives. With the click of a button, malicious actors can create content that tarnishes reputations and inflicts irreparable harm. This use of AI deepfakes, more than any other, has captured the public’s attention in recent years as the associated ethical and legal questions tie into the popular discourse surrounding systemic issues such as consent, rape culture, and violence against women.
A 2023 study noted a 550% increase in the total number of deepfake videos online in the past four years, with deepfake pornography accounting for almost 98% of those videos. Women in the entertainment industry are disproportionately affected due to their high visibility and the large volume of images, video, and audio available to them online. Of the videos analyzed in the study, 94% of the individuals targeted were women working in the entertainment industry.
As of December 2023, there is no federal law criminalizing the creation or distribution of fake pornographic images. Several states, including Texas, Minnesota, and New York, have recently passed legislation criminalizing nonconsensual deepfake pornography. Efforts to address the issue legislatively have been ongoing since 2018, with some states taking steps to criminalize nonconsensual deepfake porn.
LegitScript has observed a notable increase in merchants offering deepfakes and/or the software to produce them over the past year, and card brands appear to be on the lookout for bad actors in the industry. More legitimate operators in the industry may offer services such as AI-assisted personalized videos and other content. It is not always evident, however, whether or not a merchant has adequate controls in place to stop their users from creating deepfake and/or adult content.
Bad actors, on the other hand, operate services designed to enable users with no prior technical knowledge to generate adult content from a handful of public or private images. LegitScript is also on the lookout for merchants turning toward transaction laundering methods to obfuscate their deepfake adult services.
LegitScript Analysts Uncover Deepfake Services in Their Monitoring Work
Technological advancements often outpace regulatory frameworks, demanding nimble response and rapid adaptation from the individuals, businesses, and institutions affected and involved. LegitScript remains committed to monitoring regulatory updates and trends related to the development and dissemination of AI deepfakes and stands as a partner in identifying and reporting bad actors operating within our shared online ecosystem.
If you’re concerned about the potential effects of AI deepfake services impacting your merchant portfolio, e-commerce marketplace, or ad platform — contact us.
