AI Deepfakes Fueling Synthetic Identity Fraud and Challenges in Verification

Generative AI is revolutionizing the realm of identity theft, presenting new challenges for banks, telecommunications companies, and government agencies tasked with verifying the authenticity of individuals. Instead of simply using a stolen passport, criminals can now generate realistic faces, cloned voices, and even deepfake conversations that appear genuine.

Recent research from Coveron, an identity protection firm, in collaboration with Nord Security's threat intelligence services, highlights the growing market for what are known as "synthetic identities." These identities are not merely stolen; they are a concoction of real information acquired through data breaches, combined with fabricated names, addresses, forged documents, and AI-generated images. The result is a convincing digital persona that has never existed in reality.

According to the findings, complete synthetic identities can be purchased for as little as $200. Over the past year, researchers identified more than 10,000 posts advertising identity packages that include personal information, matching documents, and deepfake images. Notably, discussions related to deepfake technology across forums and Telegram channels monitored by the researchers surged nearly eightfold between early 2024 and mid-2026.

While these figures warrant caution—coming from a commercial cybersecurity firm and not necessarily indicating successful fraud—the threat they represent is undeniable. The U.S. Federal Reserve defines synthetic identity fraud as the use of a combination of personal information to create a fictitious person or organization for illicit financial gain. Such identities can open accounts that initially behave like legitimate ones, making them difficult for traditional fraud detection systems to identify.

The capabilities of generative AI now allow criminals to not only present stolen documents but also to engage in real-time conversations using cloned voices and faces. They can fabricate a history for these identities by creating email accounts, social media profiles, and normal-looking financial activities, ultimately enabling them to apply for loans, open bank accounts, or even receive government payments.

INTERPOL has acknowledged the rise of synthetic media as a significant issue for law enforcement, reporting that AI-generated personas have already been used to establish bank accounts and acquire mobile loans under false pretenses. In Australia, scam losses reached a staggering $2.18 billion in 2025, reflecting a 7.8% increase from the previous year. While this figure encompasses all reported scams, it underscores the expanding criminal economy that these advanced tools are infiltrating.

Challenges in Identity Verification Methods

Despite the evolving threat, Australia's identity verification processes still heavily rely on traditional methods such as document checks, photographs, video calls, and biometric comparisons. For instance, AUSTRAC's customer-identification guidelines allow businesses to compare individuals appearing in video calls with their identity document photographs. However, as technology advances, these checks can no longer be considered foolproof. If documents, faces, voices, and videos can all be generated or manipulated, multiple checks may originate from the same fraudulent source.

Australia's emerging Digital ID system aims to enhance security by verifying information against authoritative government records, reducing the need for organizations to collect and retain copies of sensitive documents. Yet, this system cannot become a single point of failure. Biometric data is particularly sensitive; unlike a compromised password, individuals cannot change their facial features or fingerprints.

To combat these challenges, banks, telecommunications companies, digital platforms, and government agencies must adopt layered verification processes, including device and behavioral analysis, ongoing transaction monitoring, and rapid human intervention when suspicious patterns arise. The Digital Duty of Care Bill in Australia seeks to shift the responsibility for online safety back onto major tech companies, emphasizing that consumers should not bear the burden of identifying AI-generated identities designed to circumvent security measures.

The Unraveling

As Australia grapples with issues of digital duty of care, privacy reform, cybersecurity, scam prevention, and AI regulation, it is crucial to address these matters as interconnected components of the broader digital trust dilemma. Continuing to regulate them in isolation risks allowing organized criminals to exploit regulatory gaps.

The era in which seeing and hearing someone served as proof of identity is rapidly fading. It is imperative that our security systems—and the responsibilities of those who operate them—evolve to meet these new challenges.