OpenAI reveals Chinese influence operation targeting dissidents through ChatGPT data

An extensive Chinese influence operation focused on the intimidation of dissidents living abroad was accidentally exposed by a Chinese law enforcement official.

The operative used the artificial intelligence tool ChatGPT to document a covert campaign of suppression, according to a report released by OpenAI.

The digital logs revealed that Chinese operators disguised themselves as United States immigration officials.

In one instance, they contacted a dissident based in the United States to claim that their public statements had violated the law. This tactic is part of a broader strategy of transnational repression.

The report by OpenAI describes an effort to use forged documents from a United States county court. These documents were used in an attempt to have the social media accounts of Chinese dissidents removed. The activity involved hundreds of operators and thousands of fake online accounts.

Ben Nimmo, the principal investigator at OpenAI, stated that this represents modern Chinese transnational repression. Nimmo noted that the effort is industrialised and targets critics of the Chinese Communist Party (CCP) across multiple platforms simultaneously. It is not limited to digital trolling.

The Chinese law enforcement official reportedly used ChatGPT as a diary to keep track of the covert network. While the operative used the AI to log activities, other tools were used to generate content that was spread through websites and social media platforms.

OpenAI confirmed it has banned the user after discovering the activity. Investigators from the company were able to match descriptions provided by the user with real-world online impacts. This included a specific effort to fake the death of a Chinese dissident.

The operative created a phony obituary and images of a gravestone which were posted online. These false rumours regarding the death of the dissident surfaced in 2023. This was corroborated by a Chinese-language report from Voice of America.

In another documented case, the operative requested the AI to draw up a plan to denigrate the incoming Prime Minister of Japan, Sanae Takaichi. The plan sought to fan online anger regarding United States tariffs on Japanese goods to damage her reputation.

Addressing the broader risks of AI manipulation at the AI Impact Summit, Prime Minister Sanae Takaichi emphasised the necessity of global cooperation. She stated that it is essential to establish a safe, secure, and trustworthy AI ecosystem to protect against such threats.

Prime Minister Takaichi noted that the key to this effort is trustworthiness. She remarked that it is important to share an interoperable ecosystem that minimises the risks posed by AI, including those related to information security. She highlighted the Hiroshima AI Process as a leading initiative.

The disclosure comes during an escalating battle between the United States and China for supremacy in artificial intelligence. The technology is increasingly viewed as a critical asset for both military operations and economic influence between the two nations.

The Pentagon is currently in a standoff with another AI firm, Anthropic, over the use of its models. Defence Secretary Pete Hegseth has set a deadline for the company to adjust safeguards or risk losing a significant government contract.

Michael Horowitz, a professor at the University of Pennsylvania and former Pentagon official, told CNN that the report demonstrates how China employs AI to enhance information operations. He noted that the competition extends to daily surveillance and information management.