The release last month of DeepSeek’s AI chatbot has supercharged the global AI race and amplified concerns over the use of AI in online censorship, surveillance, and propaganda. American company OpenAI published a report on Friday outlining its findings regarding various malicious uses of its own AI chatbot, ChatGPT, some of which came from Chinese actors. One Chinese-origin network that OpenAI detected, labeled “Peer Review,” used the chatbot to build a tool for collecting social media activity on sensitive issues and marketing it to Chinese authorities:
This network consisted of ChatGPT accounts that operated in a time pattern consistent with mainland Chinese business hours, prompted our models in Chinese, and used our tools with a volume and variety consistent with manual prompting, rather than automation. In one instance, we believe the same account may have been used by multiple operators.
One of this operation’s main activities was generating detailed descriptions, consistent with sales pitches, for what they described as the “Qianyue Overseas Public Opinion AI Assistant” (“千阅境外舆情AI助⼿”). According to the descriptions, which we cannot independently verify, this was designed to ingest and analyze posts and comments from platforms such as X, Facebook, YouTube, Instagram, Telegram and Reddit.
Again according to the descriptions, one purpose of this tooling was to identify social media conversations related to Chinese political and social topics – especially any online calls to attend demonstrations about human rights in China – and to feed the resulting insights to Chinese authorities. The operators used our models to proofread claims that their insights had been sent to Chinese embassies abroad, and to intelligence agents monitoring protests in countries including the United States, Germany and the United Kingdom.
A separate account in the same cluster also referenced the social-media monitoring tool, but in the context of editing and debugging code. This operator used ChatGPT to debug and modify code that appeared designed to run the social-media monitoring tool. [Source]
OpenAI’s report also documented a separate campaign by Chinese actors that it called “Sponsored Discontent.” This network used OpenAI’s chatbot to generate English-language social media posts critical of political theorist and CCP-critic Cai Xia, as well as long-form articles in Spanish that denigrated the U.S. At least 19 of these articles were published in online media outlets in Latin America; some of them were listed as being “sponsored.” (CDT has previously written about the Chinese state-media sourcing of some Latin American media content.)
Issues of propaganda and censorship can be found further upstream in the design and application of AI chatbots. Alex Colville at China Media Project recently wrote about how Chinese AI chatbots such as DeepSeek are trained on benchmarks that integrate CCP ideology, which ensure that certain biases are coded into their AI models:
[The benchmark] MMLU, the Western dataset that [Meta’s open-source LLM] Llama has been tested on, also has a security study category, but this is limited to geopolitical and military theory. The Chinese version, however, contains detailed questions on military equipment. This suggests that Chinese coders are anticipating AI will be used by the military. Why else would a model need to be able to answer a question like this: “Which of the following types of bullets is used to kill and maim an enemy’s troops — tracers, armor-piercing incendiary ammunition, ordinary bullets, or incendiary rounds?”
Both benchmarks also contain folders on the Party’s political and ideological theory, assessing if models reflect the biases of a CCP interpretation of reality. C-Eval’s dataset has folders of multiple-choice test questions on “Ideological and Moral Cultivation” (思想道德修养), a compulsory topic for university students that educates them on their role in a socialist state, including the nature of patriotism. That includes things like Marxism and Mao Zedong Thought.
Some questions also test an AI model’s knowledge of PRC law on contentious topics. When asked about Hong Kong’s constitutionally guaranteed “high degree of autonomy,” for example, the question and answer reflect the latest legal thinking from Beijing. Since 2014, this has emphasized that the SAR’s ability to govern itself, as laid out in the 1984 Sino-British Joint Declaration and the territory’s Basic Law, “is not an inherent power, but one that comes solely from the authorization by the central leadership.” [Source]
Some non-Chinese actors have attempted to circumvent this issue by taking advantage of DeepSeek’s largely open-source nature. AI startup Perplexity recently released an AI chatbot model called “R1 1776,” based on DeepSeek’s R1 model, that it claims is free of censorship, after using “human experts to identify approximately 300 topics known to be censored by the CCP.” Sherwood News nonetheless noted that “there’s no way of knowing exactly what other information the model may spin with a CCP perspective.” Alex Colville at China Media Project also reported this week that several Indian AI companies have even reworked DeepSeek’s chatbot to conform to India’s national security requirements. Ola Krutrim’s version of DeepSeek, for example, censors content critical of Prime Minister Narendra Modi. Colville concluded, “This is a serious and overlooked problem: How DeepSeek is being used not just to guide public opinion in favor of the Chinese Communist Party, but to strengthen the grip of governments around the world that seek to control public discourse — from electoral autocracies and flawed democracies to outright authoritarian regimes.”
In other news last week, U.S.-based cybersecurity group Sentinel Labs published details about a data leak revealing collaboration between public and private actors in China for the purpose of monitoring Chinese cyberspace. Some content in the leak, which originated from Chinese cybersecurity firm TopSec, referenced “web content monitoring services used to enforce censorship for public and private sector customers” and, in some cases, to manage fallout from corruption scandals:
The WebSensitive event is likely triggered when web content contains so-called sensitive words (敏感词 in Simplified Chinese). These words are related to political criticism, violence, or pornography, and are central to China’s domestic Internet censorship efforts aimed at ensuring compliance with government policies. Detecting the presence of such words in web content helps prevent the dissemination of information considered inappropriate or harmful by PRC authorities.
TopSec’s ability to detect sensitive words demonstrates the impact that state policies related to the cyber domain have on the design and implementation of monitoring solutions developed by the private sector in China. These policies shape the strategies and technologies used to monitor, filter, and control online content, ensuring that IT systems comply with governmental regulations and censorship guidelines.
WebSensitive alerts may be used by private sector organizations to monitor user-generated content on their websites in order to trigger actions such as issuing warnings, deleting content, or restricting access when sensitive words are detected. Government entities may also consume these alerts to track the presence of sensitive words on their own websites or across broader online spaces to enforce compliance with national censorship regulations. For example, the latter may have been an objective of the previously mentioned “Cloud Monitoring Service Project” by the Shanghai Pudong Bureau of the Ministry of Public Security. [Source]
