You've been a Baidu boy! Tech giant caught cheating on AI tests

Looks like they may need to start developing some non-artificial integrity


Baidu has been shot in its liquid metal head for cheating in a standardised and independent Artificial Intelligence test.

Hosted by Stanford University's vision lab, the Large Scale Visual Recognition Challenge (ILSVRC) saw Baidu's algorithms compete alongside those from Google, Microsoft, Apple and Facebook's FART, among others.

The Register spoke to Dr Sean Holden, senior lecturer in machine learning at Cambridge University, about the relationship between image recognition and what is classically considered AI.

"Historically, image recognition has had a very big part in artificial intelligence research, and it has taken big steps forward recently," said Dr Holden. "If you want to take an image and work out what's in it, there's loads of relevant research tucked into that, whether it's finding edges or creating Bayesian 3D models, it's quite a huge field."

"The thing is, most AI research actually isn't really working on the entire field of AI," says Dr Holden, at least not in the sense of creating general intelligence. "Human-like performance," Holden explained, "is too hard, it's too difficult to focus on."

It is hoped a technological advantage in the efficiency of an image-recognition algorithm would eventually result in a commercial advantage, though companies are still largely hovering at an error rate of around five per cent.

Baidu had claimed to have scored a record-low error rate in the image recognition tests of 4.58 per cent. Google, in comparison, hit a 4.8 per cent error rate, while Microsoft achieved 4.94 per cent.

The ILSVRC organisers announced in May that one group had circumvented their testing policy by creating and using multiple accounts to run more evaluations every week than their competitors. An updated note identifies this group as Baidu.

A statement from Baidu's Heterogeneous Computing team states that "recently the ILSVRC organizers contacted [us] to inform us that we exceeded the allowable number of weekly submissions to the ImageNet servers (~ 200 submissions during the lifespan of our project)."

"We apologize for this mistake and are continuing to review the results. We have added a note to our research paper, Deep Image: Scaling up Image Recognition [arxiv], and will continue to provide relevant updates as we learn more."

The apology, which is directed at the ILSVRC community, acknowledges "the mistake" and claims they are "staunch supporters of fairness and transparency in the ImageNet Challenge and are committed to the integrity of the scientific process".

With the arena of image recognition to one side, Dr Holden explained developments in Skynet-style AI to us. "Basically, general intelligence is a very long way away. The more you learn about neuroscience, about how the human brain works – it's not happening in my lifetime. In a long enough timeline, yes, but I'm not holding my breath." ®

Similar topics

Broader topics


Other stories you might like

  • Cheers ransomware hits VMware ESXi systems
    Now we can say extortionware has jumped the shark

    Another ransomware strain is targeting VMware ESXi servers, which have been the focus of extortionists and other miscreants in recent months.

    ESXi, a bare-metal hypervisor used by a broad range of organizations throughout the world, has become the target of such ransomware families as LockBit, Hive, and RansomEXX. The ubiquitous use of the technology, and the size of some companies that use it has made it an efficient way for crooks to infect large numbers of virtualized systems and connected devices and equipment, according to researchers with Trend Micro.

    "ESXi is widely used in enterprise settings for server virtualization," Trend Micro noted in a write-up this week. "It is therefore a popular target for ransomware attacks … Compromising ESXi servers has been a scheme used by some notorious cybercriminal groups because it is a means to swiftly spread the ransomware to many devices."

    Continue reading
  • Twitter founder Dorsey beats hasty retweet from the board
    As shareholders sue the social network amid Elon Musk's takeover scramble

    Twitter has officially entered the post-Dorsey age: its founder and two-time CEO's board term expired Wednesday, marking the first time the social media company hasn't had him around in some capacity.

    Jack Dorsey announced his resignation as Twitter chief exec in November 2021, and passed the baton to Parag Agrawal while remaining on the board. Now that board term has ended, and Dorsey has stepped down as expected. Agrawal has taken Dorsey's board seat; Salesforce co-CEO Bret Taylor has assumed the role of Twitter's board chair. 

    In his resignation announcement, Dorsey – who co-founded and is CEO of Block (formerly Square) – said having founders leading the companies they created can be severely limiting for an organization and can serve as a single point of failure. "I believe it's critical a company can stand on its own, free of its founder's influence or direction," Dorsey said. He didn't respond to a request for further comment today. 

    Continue reading
  • Snowflake stock drops as some top customers cut usage
    You might say its valuation is melting away

    IPO darling Snowflake's share price took a beating in an already bearish market for tech stocks after filing weaker than expected financial guidance amid a slowdown in orders from some of its largest customers.

    For its first quarter of fiscal 2023, ended April 30, Snowflake's revenue grew 85 percent year-on-year to $422.4 million. The company made an operating loss of $188.8 million, albeit down from $205.6 million a year ago.

    Although surpassing revenue expectations, the cloud-based data warehousing business saw its valuation tumble 16 percent in extended trading on Wednesday. Its stock price dived from $133 apiece to $117 in after-hours trading, and today is cruising back at $127. That stumble arrived amid a general tech stock sell-off some observers said was overdue.

    Continue reading
  • Amazon investors nuke proposed ethics overhaul and say yes to $212m CEO pay
    Workplace safety, labor organizing, sustainability and, um, wage 'fairness' all struck down in vote

    Amazon CEO Andy Jassy's first shareholder meeting was a rousing success for Amazon leadership and Jassy's bank account. But for activist investors intent on making Amazon more open and transparent, it was nothing short of a disaster.

    While actual voting results haven't been released yet, Amazon general counsel David Zapolsky told Reuters that stock owners voted down fifteen shareholder resolutions addressing topics including workplace safety, labor organizing, sustainability, and pay fairness. Amazon's board recommended voting no on all of the proposals.

    Jassy and the board scored additional victories in the form of shareholder approval for board appointments, executive compensation and a 20-for-1 stock split. Jassy's executive compensation package, which is tied to Amazon stock price and mostly delivered as stock awards over a multi-year period, was $212 million in 2021. 

    Continue reading
  • Confirmed: Broadcom, VMware agree to $61b merger
    Unless anyone out there can make a better offer. Oh, Elon?

    Broadcom has confirmed it intends to acquire VMware in a deal that looks set to be worth $61 billion, if it goes ahead: the agreement provides for a “go-shop” provision under which the virtualization giant may solicit alternative offers.

    Rumors of the proposed merger emerged earlier this week, amid much speculation, but neither of the companies was prepared to comment on the deal before today, when it was disclosed that the boards of directors of both organizations have unanimously approved the agreement.

    Michael Dell and Silver Lake investors, which own just over half of the outstanding shares in VMware between both, have apparently signed support agreements to vote in favor of the transaction, so long as the VMware board continues to recommend the proposed transaction with chip designer Broadcom.

    Continue reading

Biting the hand that feeds IT © 1998–2022