You've been a Baidu boy! Tech giant caught cheating on AI tests

Looks like they may need to start developing some non-artificial integrity


Baidu has been shot in its liquid metal head for cheating in a standardised and independent Artificial Intelligence test.

Hosted by Stanford University's vision lab, the Large Scale Visual Recognition Challenge (ILSVRC) saw Baidu's algorithms compete alongside those from Google, Microsoft, Apple and Facebook's FART, among others.

The Register spoke to Dr Sean Holden, senior lecturer in machine learning at Cambridge University, about the relationship between image recognition and what is classically considered AI.

"Historically, image recognition has had a very big part in artificial intelligence research, and it has taken big steps forward recently," said Dr Holden. "If you want to take an image and work out what's in it, there's loads of relevant research tucked into that, whether it's finding edges or creating Bayesian 3D models, it's quite a huge field."

"The thing is, most AI research actually isn't really working on the entire field of AI," says Dr Holden, at least not in the sense of creating general intelligence. "Human-like performance," Holden explained, "is too hard, it's too difficult to focus on."

It is hoped a technological advantage in the efficiency of an image-recognition algorithm would eventually result in a commercial advantage, though companies are still largely hovering at an error rate of around five per cent.

Baidu had claimed to have scored a record-low error rate in the image recognition tests of 4.58 per cent. Google, in comparison, hit a 4.8 per cent error rate, while Microsoft achieved 4.94 per cent.

The ILSVRC organisers announced in May that one group had circumvented their testing policy by creating and using multiple accounts to run more evaluations every week than their competitors. An updated note identifies this group as Baidu.

A statement from Baidu's Heterogeneous Computing team states that "recently the ILSVRC organizers contacted [us] to inform us that we exceeded the allowable number of weekly submissions to the ImageNet servers (~ 200 submissions during the lifespan of our project)."

"We apologize for this mistake and are continuing to review the results. We have added a note to our research paper, Deep Image: Scaling up Image Recognition [arxiv], and will continue to provide relevant updates as we learn more."

The apology, which is directed at the ILSVRC community, acknowledges "the mistake" and claims they are "staunch supporters of fairness and transparency in the ImageNet Challenge and are committed to the integrity of the scientific process".

With the arena of image recognition to one side, Dr Holden explained developments in Skynet-style AI to us. "Basically, general intelligence is a very long way away. The more you learn about neuroscience, about how the human brain works – it's not happening in my lifetime. In a long enough timeline, yes, but I'm not holding my breath." ®

Similar topics


Other stories you might like

  • Want to buy your own piece of the Pi? No 'urgency' says Upton of the listing rumours

    A British success story... what happens next?

    Industry talk is continuing to circulate regarding a possible public listing of the UK makers of the diminutive Raspberry Pi computer.

    Over the weekend, The Telegraph reported that a spring listing could be in the offing, with a valuation of more than £370m.

    Pi boss, Eben Upton, described the newspaper's article as "interesting" in an email to The Register today, before repeating that "we're always looking at ways to fund the future growth of the business, but the $45m we raised in September has taken some of the urgency out of that."

    Continue reading
  • JetBrains embraces remote development with new IDE for multiple programming languages

    Security, collaboration, flexible working: Fleet does it all apparently

    JetBrains has introduced remote development for its range of IDEs as well as previewing a new IDE called Fleet, which will form the basis for fresh tools covering all major programming languages.

    JetBrains has a core IDE used for the IntelliJ IDEA Java tool as well other IDEs such as Android Studio, the official programming environment for Google Android, PyCharm for Python, Rider for C#, and so on. The IDEs run on the Java virtual machine (JVM) and are coded using Java and Kotlin, the latter being primarily a JVM language but with options for compiling to JavaScript or native code.

    Fleet is "both an IDE and a lightweight code editor," said the company in its product announcement, suggesting perhaps that it is feeling some pressure from the success of Microsoft's Visual Studio Code, which is an extensible code editor. Initial language support is for Java, Kotlin, Go, Python, Rust, and JavaScript, though other languages such as C# will follow. Again like VS Code, Fleet can run on a local machine or on a remote server. The new IDE uses technology developed for IntelliJ such as its code-processing engine for features such as code completion and refactoring.

    Continue reading
  • Nextcloud and cloud chums fire off competition complaint to the EU over Microsoft bundling OneDrive with Windows

    No, it isn't the limited levels of storage that have irked European businesses

    EU software and cloud businesses have joined Nextcloud in filing a complaint with the European Commission regarding Microsoft's alleged anti-competitive behaviour over the bundling of its OS with online services.

    The issue is OneDrive and Microsoft's habit of packaging it (and other services such as Teams) with Windows software.

    Nextcloud sells on-premises collaboration platforms that it claims combine "the convenience and ease of use of consumer-grade solutions like Dropbox and Google Drive with the security, privacy and control business needs." Microsoft's cloud storage system, OneDrive, is conspicuous by its absence.

    Continue reading

Biting the hand that feeds IT © 1998–2021