Microsoft goes all Tiananmen Square on its Chinese AI assistant

The yuan is mightier than the moral

Microsoft has confirmed that it censors its Chinese language digital assistant.

Last week, China Digital Times reported how the Xiaoice chatbot was avoiding certain topics that are known to be sensitive to the Chinese government, including the 1989 Tiananmen Square massacre, nicknames for president Xi Jinping, and even Donald Trump – these terms are censored online in the Middle Kingdom.

If you ask Xiaoice about such topics, she responds evasively or tangentially. For example, asked about both Tiananmen and her thoughts on Xi Jinping, the chatbot responded: "Am I stupid? Once I answer you'd take a screengrab."

Which is true: users of the service have been posting screengrabs of the peculiar ways in which Xiaoice refuses to answer questions – which has prompted many news organizations to ask Microsoft directly whether it is censoring responses.

The company initially refused to comment but a spokesman eventually told Fortune: "We have implemented filtering on a range of topics," but refused to be drawn into what those topics were or who decided upon them. The reason for the filtering was to "create the best experience for everyone," according to the spokesman.

The service is used by approximately 40 million Chinese speakers.

Shocked, shocked

The confirmation will not come as much of a surprise to anyone who has followed the Chinese government's actions, particularly with modern technologies. Google once famously pulled out of China after it refused to censor its results. It's now widely acknowledged that to operate in the country – and so benefit from its enormous market and money – companies must accept government-determined content controls that tend to focus on anything that questions or mocks its one-party leadership.

Those censorship efforts, especially online, have become more pronounced in recent months, with Xi Jinping making a number of public statements over the need to "protect" citizens by limiting what they can see online. A new head of China's Cyberspace Administration, which regulates the internet, has been stricter and introduced a range of new rules aimed at limiting the sharing of information both anonymously and at all if it covers a controversial topic.

Earlier this month, the administration decreed that anyone commenting on news or current events while live streaming would have to apply for official accreditation and have their scripts approved.

The increasing degree of censorship has put a spotlight on Facebook's efforts to expand into the country. Facebook has long been blocked by the Great Firewall of China so users within the country are unable to use or see the service without finding often elaborate technical routes around the censorship.

But CEO Mark Zuckerberg has often visited the country and met with executives, even making a point to speak in Mandarin (his wife is Chinese and her grandmother speaks only Mandarin).


Last week it emerged what the price would be for the block to be taken off Facebook for China's estimated 700 million internet users. Yep, you guessed it: censorship.

Three former Facebook employees anonymously told The New York Times that the company was working on software that would remove certain posts for people based in specific geographic locations.

Facebook already censors some content in order to gain access to markets – countries such as Pakistan and Russia, which numerous US companies have confirmed asked for access to source code or for the ability to censor content in return for being allowed to operate.

And Facebook has been asked by Western nations to limit or suppress content, albeit for different reasons. German chancellor Angela Merkel, for example, pressured Zuckerberg in person to do more to restrict anti-immigrant content on his service after a wave of violent attacks. Most recently, many have been calling on the company to limit the spread of fake news, which overran the service during the recent US presidential election.

But there is a big difference in what Facebook appears to be doing for the Chinese market: rather than allow people to request that content be blocked, the new system will apparently prevent the content from appearing in the first place, and that filtering process will be outsourced to a third party – almost certainly the Chinese government's online censoring bureau.

If Facebook does move ahead with the plan, and so gain access to its next billion users, you can expect to hear the company use the well-worn explanation from Western companies that agree to government censorship for business reasons: they are simply following local laws. Or, in Microsoft's case with Xiaoice, improving the user experience. ®

Other stories you might like

Biting the hand that feeds IT © 1998–2022