Montezuma's Revenge still too tough for AI, new Google Brain office, and other bits and bytes

A wonderful week in machine learning

Roundup Hello, here are some quick AI announcements from this week. A researcher reminds us to be wary of the hype around Montezuma's Revenge, there are some new framework updates from Google and Microsoft, and a new Google Brain office in Amsterdam.

Montezuma’s Revenge isn’t solved yet: OpenAI and DeepMind researchers have boasted about achieving record scores on the old Atari game using AI agents, but the challenge isn’t over.

In a blog post, Arthur Juliani, a reinforcement learning researcher from Unity, explains why Montezuma’s Revenge is such a difficult game to teach machines. The rewards are sparse, and although OpenAI and DeepMind have reached decent scores it’s because both methods rely on human imitation rather than learning.

Both DeepMind and OpenAI relied on training bots to play the game by copying human gameplay in videos. By learning through imitation, the agent is more likely able to memorize good sequences of moves and would not achieve the same high scores if it had to learn from scratch.

“Rather than developing a general-purpose solution to game playing (as the two DeepMind papers titles suggest), what has really been developed is an intelligent method for exploiting a key weakness in Montezuma’s Revenge as an experimental platform: its determinism,” he explained.

“Every time a human or agent plays Montezuma’s Revenge, they are presented with the exact same set of rooms, each containing the exact same set of obstacles and puzzles. As such, the simple memorization of the movements through each room is enough to lead to a high-score, and the ability to complete the level.”

So, don’t believe all the hype out there.

New TensorFlow goodies: TensorFlow 1.9.0 is out and includes new features to work more seamlessly with the Keras API.

It also has an improved interface for Python, and has fixed some bugs and added some new Estimator models. Download it here.

Google Brain Amsterdam: Google Brain has launched a new office in Amsterdam.

It has started looking for research scientists to work in natural language processing, search, hardware, mobile compilers - you name it.

“Much of our work is best understood as part of the 'deep learning' subfield of machine learning, but we are interested in any methods — such as evolutionary computing, novelty search or reinforcement learning — that advance the capabilities of machine intelligence. We have resources and access to projects impossible to find elsewhere,” it boasted in a job advert.

Some researchers from OpenAI and DeepMind including Tim Salimans, Durk Kingma, Nal Kalchbrenner, and Lasse Espeholt have joined so far.

Reversible generative models: We’re obsessed with using machine learning to augment faces with filters. OpenAI has released Glow, a model that can easily blend two faces together or alter certain attributes and then reverse all the changes.

You can manipulate Glow to make a person smile more or less, look older or younger, give them blonder hair, narrower or wider eyes, and even whack on a beard. The more amusing option creates a series of images that increasingly maps one person’s face onto another’s. All these changes can be reverted to retrieve the original input.

It’s all down to clever encoding. The inputs are encoded and an average latent vector between the inputs is computed between different attributes, whether it be two people’s faces or making the hair blonder.

The vector direction between the two can be manipulated to add as little or as much of a certain a feature as you like. The idea has been around for a while, and researchers at OpenAI have made their model more efficient by simplifying the architecture.

Glow was trained on 30,000 images of faces using five machines with eight GPUs each. The inference stage takes about 130 milliseconds to generate a 256x256 image using an Nvidia 1080 Ti card.

You can play around with the model here, or go straight to the research and code here.

Updated Microsoft framework: Microsoft Research has launched ML.Net.03, the framework now allows users to export models to run on Windows 10 devices.

It supports ONNX (Open Neural Network Exchange), an open-source collaboration between software and hardware companies like Facebook, Amazon, Microsoft, ARM and Huawei. The goal is to make it easier for developers to transfer models written in different frameworks like Caffe 2, PyTorch, or Microsoft’s Cognitive Toolkit.

“With this release of ML.NET v0.3 you can export certain ML.NET models to the ONNX-ML format. ONNX models can be used to infuse machine learning capabilities in platforms like Windows ML which evaluates ONNX models natively on Windows 10 devices taking advantage of hardware acceleration,” it said.

The third iteration of ML.Net also includes new features to make it easier to include binary and multiclass classification and regression algorithms. The first version of the framework was introduced at its Build conference this year.

You can read more about the updates and see code samples here. ®

Broader topics

Other stories you might like

  • Your snoozing iOS 15 iPhone may actually be sleeping with one antenna open
    No, you're not really gonna be hacked. But you may be surprised

    Some research into the potentially exploitable low-power state of iPhones has sparked headlines this week.

    While pretty much no one is going to utilize the study's findings to attack Apple users in any meaningful way, and only the most high-profile targets may find themselves troubled by all this, it at least provides some insight into what exactly your iOS handheld is up to when it's seemingly off or asleep. Or none of this is news to you. We'll see.

    According to the research, an Apple iPhone that goes asleep into low-power mode or is turned off isn't necessarily protected against surveillance. That's because some parts of it are still operating at low power.

    Continue reading
  • China will produce one in five of the chips it uses in 2026, says analyst
    Well short of planned 70 percent domestic capacity

    China’s integrated circuit (IC) production has failed to keep pace with its appetite for silicon, with market research firm IC Insights predcicting the nation will produce only one in five ICs it uses in 2026.

    That figure is a increase from 2021's one in six, and reflects eight percent compound annual growth rate from 2021 to 2026. But it means China will miss its own targets for locally-made-and-consumed silicon.

    “Although China has been the largest consuming country for ICs since 2005, it does not necessarily mean that large increases in IC production within China would immediately follow, or ever follow” said the firm in a bulletin on Wednesday.

    Continue reading
  • Tencent happily parting ways with loss-making cloud customers
    Cutting costs across sprawling business as COVID makes life hard in China

    Chinese tech giant Tencent has recorded its first ever quarter-to-quarter revenue fall, warned that COVID-19 lockdowns will hurt messing with its business, and cautioned against assumptions that Beijing is ready to enthusiastically support tech companies.

    On its Q1 2022 earnings call yesterday, the company offered more explanation of its shifting cloud strategy.

    Chief strategy officer James Mitchell told investors the company is pleased to have shown loss-making cloud customers the door, and “proactively scaled back … deeply discounted infrastructure-only contracts for basic services such as cloud compute and content delivery network.” Projects that had high costs and/or relied on sub-contractors have also been scaled back.

    Continue reading
  • Will this be one of the world's first RISC-V laptops?
    A sneak peek at a notebook that could be revealed this year

    Pic As Apple and Qualcomm push for more Arm adoption in the notebook space, we have come across a photo of what could become one of the world's first laptops to use the open-source RISC-V instruction set architecture.

    In an interview with The Register, Calista Redmond, CEO of RISC-V International, signaled we will see a RISC-V laptop revealed sometime this year as the ISA's governing body works to garner more financial and development support from large companies.

    It turns out Philipp Tomsich, chair of RISC-V International's software committee, dangled a photo of what could likely be the laptop in question earlier this month in front of RISC-V Week attendees in Paris.

    Continue reading
  • Did hoodwink Americans with IRS facial-recognition tech, senators ask
    Biz tells us: Won't someone please think of the ... fraud we've stopped

    Democrat senators want the FTC to investigate "evidence of deceptive statements" made by regarding the facial-recognition technology it controversially built for Uncle Sam. made headlines this year when the IRS said US taxpayers would have to enroll in the startup's facial-recognition system to access their tax records in the future. After a public backlash, the IRS reconsidered its plans, and said taxpayers could choose non-biometric methods to verify their identity with the agency online.

    Just before the IRS controversy, said it uses one-to-one face comparisons. "Our one-to-one face match is comparable to taking a selfie to unlock a smartphone. does not use one-to-many facial recognition, which is more complex and problematic. Further, privacy is core to our mission and we do not sell the personal information of our users," it said in January.

    Continue reading
  • Meet Wizard Spider, the multimillion-dollar gang behind Conti, Ryuk malware
    Russia-linked crime-as-a-service crew is rich, professional – and investing in R&D

    Analysis Wizard Spider, the Russia-linked crew behind high-profile malware Conti, Ryuk and Trickbot, has grown over the past five years into a multimillion-dollar organization that has built a corporate-like operating model, a year-long study has found.

    In a technical report this week, the folks at Prodaft, which has been tracking the cybercrime gang since 2021, outlined its own findings on Wizard Spider, supplemented by info that leaked about the Conti operation in February after the crooks publicly sided with Russia during the illegal invasion of Ukraine.

    What Prodaft found was a gang sitting on assets worth hundreds of millions of dollars funneled from multiple sophisticated malware variants. Wizard Spider, we're told, runs as a business with a complex network of subgroups and teams that target specific types of software, and has associations with other well-known miscreants, including those behind REvil and Qbot (also known as Qakbot or Pinkslipbot).

    Continue reading
  • Supreme Court urged to halt 'unconstitutional' Texas content-no-moderation law
    Everyone's entitled to a viewpoint but what's your viewpoint on what exactly is and isn't a viewpoint?

    A coalition of advocacy groups on Tuesday asked the US Supreme Court to block Texas' social media law HB 20 after the US Fifth Circuit Court of Appeals last week lifted a preliminary injunction that had kept it from taking effect.

    The Lone Star State law, which forbids large social media platforms from moderating content that's "lawful-but-awful," as advocacy group the Center for Democracy and Technology puts it, was approved last September by Governor Greg Abbott (R). It was immediately challenged in court and the judge hearing the case imposed a preliminary injunction, preventing the legislation from being enforced, on the basis that the trade groups opposing it – NetChoice and CCIA – were likely to prevail.

    But that injunction was lifted on appeal. That case continues to be litigated, but thanks to the Fifth Circuit, HB 20 can be enforced even as its constitutionality remains in dispute, hence the coalition's application [PDF] this month to the Supreme Court.

    Continue reading
  • How these crooks backdoor online shops and siphon victims' credit card info
    FBI and co blow lid off latest PHP tampering scam

    The FBI and its friends have warned businesses of crooks scraping people's credit-card details from tampered payment pages on compromised websites.

    It's an age-old problem: someone breaks into your online store and alters the code so that as your customers enter their info, copies of their data is siphoned to fraudsters to exploit. The Feds this week have detailed one such effort that reared its head lately.

    As early as September 2020, we're told, miscreants compromised at least one American company's vulnerable website from three IP addresses: 80[.]249.207.19, 80[.]82.64.211 and 80[.]249.206.197. The intruders modified the web script TempOrders.php in an attempt to inject malicious code into the checkout.php page.

    Continue reading

Biting the hand that feeds IT © 1998–2022