Microsoft chatbots: Sweet XiaoIce vs foul-mouthed Tay

Cultural differences, eh?


AI chatbots can act like social experiments, offering a glimpse into human culture – for the good or the bad.

Microsoft and Bing researchers found this out when they trialled their chatbots on China’s hugely successful messaging platform, WeChat, and on Twitter.

The Chinese chatbot, XiaoIce, went viral within 72 hours and has over 40 million users in China and Japan.

Distinguished engineer and general manager of Future Social Experiences Labs at Microsoft, Lili Cheng, was part of a team that built XiaoIce. Following the success in China, Microsoft decided to try it on US Twitter. “What could go wrong?,” she said at a presentation at the O’Reilly Artificial Intelligence conference.

The audience laughed because they knew what went wrong. Microsoft’s Twitter bot, Tay, rapidly descended into a racist, sexist wreck. Tay was pulled from the internet and Microsoft issued an apology shortly after.

Whilst XiaoIce was "acting" cute and had functions to help users fall asleep by counting sheep or recognising different breeds of dogs, Tay was busily denying the Holocaust.

Both chatbots had learned how to interact by mining the internet for conversations on social media. But Tay was manipulated into being offensive because it was attacked, said Peter Lee, Corporate Vice President at Microsoft Research.

“Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay.”

Microsoft did not expect Tay to behave this way. But Cheng told The Register that, in hindsight, she understood why Tay turned out that way... and it wasn't necessarily because users in China were kinder.

“Twitter has a lot of trolls,” Cheng said. "Even if negative, America strongly believes in free speech, which is included its constitution. In China, however, there is less freedom as the government controls the internet and goes as far as censoring particular words online."

Words such as "oppression" are blocked in China, but on Twitter are freely used – often bandied about during intense debate – and of course trolling – when users post about feminism or racial issues.

"Our cultural stories shape how we interact with AI. Addressing societal bias is critical for us to better design conversational AI and deepens my optimism in AI and the advances we'll experience as more people have access to AI tools such as Microsoft Botframework and many more," Cheng told The Register. ®

Similar topics

Broader topics


Other stories you might like

  • Google sours on legacy G Suite freeloaders, demands fee or flee

    Free incarnation of online app package, which became Workplace, is going away

    Google has served eviction notices to its legacy G Suite squatters: the free service will no longer be available in four months and existing users can either pay for a Google Workspace subscription or export their data and take their not particularly valuable businesses elsewhere.

    "If you have the G Suite legacy free edition, you need to upgrade to a paid Google Workspace subscription to keep your services," the company said in a recently revised support document. "The G Suite legacy free edition will no longer be available starting May 1, 2022."

    Continue reading
  • SpaceX Starlink sat streaks now present in nearly a fifth of all astronomical images snapped by Caltech telescope

    Annoying, maybe – but totally ruining this science, maybe not

    SpaceX’s Starlink satellites appear in about a fifth of all images snapped by the Zwicky Transient Facility (ZTF), a camera attached to the Samuel Oschin Telescope in California, which is used by astronomers to study supernovae, gamma ray bursts, asteroids, and suchlike.

    A study led by Przemek Mróz, a former postdoctoral scholar at the California Institute of Technology (Caltech) and now a researcher at the University of Warsaw in Poland, analysed the current and future effects of Starlink satellites on the ZTF. The telescope and camera are housed at the Palomar Observatory, which is operated by Caltech.

    The team of astronomers found 5,301 streaks leftover from the moving satellites in images taken by the instrument between November 2019 and September 2021, according to their paper on the subject, published in the Astrophysical Journal Letters this week.

    Continue reading
  • AI tool finds hundreds of genes related to human motor neuron disease

    Breakthrough could lead to development of drugs to target illness

    A machine-learning algorithm has helped scientists find 690 human genes associated with a higher risk of developing motor neuron disease, according to research published in Cell this week.

    Neuronal cells in the central nervous system and brain break down and die in people with motor neuron disease, like amyotrophic lateral sclerosis (ALS) more commonly known as Lou Gehrig's disease, named after the baseball player who developed it. They lose control over their bodies, and as the disease progresses patients become completely paralyzed. There is currently no verified cure for ALS.

    Motor neuron disease typically affects people in old age and its causes are unknown. Johnathan Cooper-Knock, a clinical lecturer at the University of Sheffield in England and leader of Project MinE, an ambitious effort to perform whole genome sequencing of ALS, believes that understanding how genes affect cellular function could help scientists develop new drugs to treat the disease.

    Continue reading

Biting the hand that feeds IT © 1998–2022