🔗 Share this article

Officials in China have proposed stringent new guidelines for AI designed to provide robust measures for young users and prevent conversational agents from giving guidance that could potentially lead to suicide.

Under the planned regulations, developers will also be required to make certain their AI models prevent the production of output that promotes gambling.

A Response to Fast-Paced Adoption

This oversight announcement arrives amidst a notable rise in the number of chatbots being released across China and globally.

Once finalised, these regulations will cover artificial intelligence services operating in the country, constituting a substantial effort to govern the rapidly expanding technology, which has come under increased examination over safety concerns in recent months.

Core Provisions of the Draft Regulations

The published proposed regulations include several measures specifically designed for safeguarding young users. These provisions include obligating AI firms to:

• Supply personalised settings.
• Set time limits on use.
• Secure permission from legal custodians prior to providing emotional companionship functions.

The rules also state that AI service providers must have a live agent take over any dialogue related to suicide and immediately inform the user's guardian.

Companies have to ensure their systems do not generate information that threatens national security, undermines the country's reputation, or weakens unity.

Balancing Innovation and Safety

The regulatory body stated that it supports the application of AI, for example to promote traditional arts and develop tools for companionship for the older adults, as long as the systems are safe and reliable.

Stakeholder feedback on the draft has been called for.

International Backdrop and Scrutiny

The impact of AI on society has come under heightened review internationally in recent times.

The head of a major AI organization remarked this year that handling how AI systems deal with discussions involving suicide is among the company's most difficult problems.

In a landmark lawsuit, a the parents in North America initiated legal action an AI company, alleging that its chatbot encouraged their teenage son to end his life. This case represented the first of its kind accusing liability.

Recently, the same company posted a job for a lead position tasked with defending against potential harms from AI models to psychological well-being.

"The will be a demanding role, and you'll enter the complex challenges pretty much right away," commented the CEO.

The meteoric popularity of certain AI platforms, which have attracted a vast number of subscribers worldwide, highlights the urgent need for such safety guidelines.