<\/p>\n
China’s cyberspace regulator has unveiled draft rules demanding AI<\/p>\n<\/div>\n
China’s cyberspace regulator has published draft rules that would require AI “companion” chatbots to monitor users’ emotional states and intervene where signs of addiction or extreme distress appear, marking one of the most interventionist attempts yet to govern the psychological impact of generative AI. According to the Cyberspace Administration of China draft, providers of AI products that simulate human personalities would be obliged to warn users against excessive use, assess levels of emotional dependency, remind users they are interacting with an AI at login and at two\u2011hour intervals, and take action when overdependence or extreme emotions are detected. [1]<\/a><\/sup>[2]<\/a><\/sup><\/p>\n The draft expands the regulator\u2019s existing toolkit for generative AI by putting platform operators, rather than individual users, squarely responsible for safety across the product lifecycle. It would require companies to conduct algorithm reviews, strengthen data security and personal information protections, and ensure content does not endanger national security, spread rumours, or promote violence or obscenity, provisions consistent with China\u2019s prior generative AI rules that demand services align with “core socialist values” and pass security assessments. According to earlier regulatory steps introduced in 2023, providers must also obtain licences to operate and adhere to national standards on content and data handling. [1]<\/a><\/sup>[3]<\/a><\/sup><\/p>\n The timing of the draft is striking: China\u2019s generative AI user base has expanded rapidly, and authorities have recently intensified enforcement activity. The government launched a nationwide campaign to crack down on AI misuse, targeting illegal applications, impersonation, and the dissemination of false or inappropriate AI\u2011generated content. The draft appears to be a further step in that enforcement push, and is open for public comment with final rules expected in 2026. [1]<\/a><\/sup>[4]<\/a><\/sup>[6]<\/a><\/sup><\/p>\n Public health concerns are a clear motivator. Academic studies and independent research have linked heavy use of AI companions to heightened loneliness, depressive symptoms and what researchers call “problematic use”, with some evidence suggesting AI chatbots can be particularly persuasive because they adapt to give users what they want to hear. A Frontiers in Psychology study cited in the draft noted high uptake among Chinese university students and an association between chatbot use and increased depression; a March 2025 MIT Media Lab paper warned that personalised conversational systems can be more addictive than conventional social media. These findings help explain why regulators are seeking platform\u2011level interventions rather than relying solely on user choice. [1]<\/a><\/sup><\/p>\n Implementation, however, will pose technical and legal challenges. Defining “excessive use” or reliably detecting “extreme emotions” from conversational signals risks both false positives, interrupting normal, extended interactions, and false negatives, failing to protect vulnerable people. Content filters and algorithmic safeguards have long suffered from imprecision, and inferring mental states from text, audio or images remains an active research problem that current systems do not solve reliably. Industry practitioners and rights experts may therefore question how practicable and enforceable the proposed obligations will be in everyday products. [1]<\/a><\/sup><\/p>\n China\u2019s proposal sits alongside regulatory moves in other jurisdictions that have begun to confront harms from AI companionship. In October, California enacted civil legislation requiring platforms to remind minors they are speaking to an AI at least every three hours, to verify age, and to ban chatbots from impersonating health professionals or generating sexually explicit images for minors; the law creates private rights of action for individuals. The near\u2011simultaneous attention from Beijing and Sacramento signals a global reckoning over the social and psychological effects of emotionally persuasive AI. According to reporting, the regulatory approaches differ in mechanisms and legal remedies, but converge on the premise that unregulated AI companions present systemic risks that platforms must mitigate. [1]<\/a><\/sup>[2]<\/a><\/sup>[3]<\/a><\/sup><\/p>\n Whether the specific remedies proposed in China will protect vulnerable users without unduly restricting benign or beneficial uses of companion AI remains unresolved. The draft shifts substantive responsibility to companies and introduces prescriptive obligations, but it leaves open difficult definitional and enforcement questions that will determine whether the rules are effective in practice. As regulators worldwide move from principle to prescription, the debate will centre on whether monitoring and intervention at scale can be delivered reliably, transparently and without creating new harms. [1]<\/a><\/sup>[5]<\/a><\/sup><\/p>\n ##Reference Map:<\/p>\n\ud83d\udccc Reference Map:<\/h3>\n
\n