OpenAI has uncovered significant misuse of its AI tools, especially ChatGPT, by groups linked to the Chinese government engaged in misinformation and covert influence campaigns globally. In a report released Thursday, OpenAI revealed it disrupted several such operations in the past three months by banning involved accounts.
Ben Nimmo, OpenAI’s lead investigator, highlighted that these campaigns use a growing variety of tactics, all violating OpenAI’s usage policies. The report detailed four separate campaigns likely backed by Chinese groups, each with unique objectives but sharing common misuse of AI.
One campaign, dubbed Sneer Review, used ChatGPT to generate fake posts and comments on platforms like TikTok, X (formerly Twitter), Reddit, and Facebook. It aimed to create artificial online engagement on topics ranging from criticism of a Taiwan-based video game to mixed reactions about the closure of the US Agency for International Development (USAID). The operation’s sophistication lay in simulating natural conversations by producing both initial posts and replies, misleading observers about the authenticity of the discussions.
Another striking misuse involved ChatGPT writing internal performance reviews of the influence campaigns themselves — detailed documents outlining how these operations were run. OpenAI confirmed that the online behaviors they observed matched the contents of these AI-generated internal reports.
Additional operations included leveraging ChatGPT to assist cyber activities: modifying scripts, configuring systems, and developing brute-force password tools. There were also intelligence-gathering attempts where fake AI-generated personas posed as journalists or analysts to interact with real users and extract information. In one case, AI-generated content was linked to communication concerning a US Senator’s correspondence, though OpenAI could not confirm if it was actually sent.
Since ChatGPT’s launch in late 2022, concerns about generative AI enabling misinformation have intensified. OpenAI, valued recently at $300 billion after new funding rounds, faces increasing pressure to curb abuse of its powerful tools while advancing AI innovation.
