Google's DeepMind says its AI coding bot is 'competitive' with humans

Gulp! Code that codes displays some critical thinking capabilities


Alphabet-owned AI outfit DeepMind claims it has created an AI that can write programming code, find novel solutions to interesting problems, and do it at the level of the mid-ranking human entrants in coding contests.

Dubbed "AlphaCode" and detailed in a pre-print paper [PDF], the tool is said to advance on previous automated coding efforts by displaying the ability to tackle "problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding."

Previous efforts to create code that codes haven't been able to reach that level of sophistication, but have done decently when asked to handle simple maths or programming chores.

DeepMind's paper explains that AlphaCode's developers addressed three important elements:

  • Finding a clean dataset to work with, and as coding contests are not hard to find that data was easily secured.
  • Designing an appropriate algorithm, along the lines of the transformer-based architectures used for natural language processing or image recognition.
  • Creating many sample solutions, then filtering them to find work that would be useful for the problem at hand.

That problem was participating in coding contests run by Codeforces.

One of the challenges AlphaCode took on is described as follows:

You are given two strings s and t, both consisting of lowercase English letters. You are going to type the string s character by character, from the first character to the last one.

When typing a character, instead of pressing the button corresponding to it, you can press the Backspace button. It deletes the last character you have typed among those that aren't deleted yet (or does nothing if there are no characters in the current string). For example, if s is "abcbd" and you press Backspace instead of typing the first and the fourth characters, you will get the string "bd" (the first press of Backspace deletes no character, and the second press deletes the character "c"). Another example, if s is "abcaa" and you press Backspace instead of the last two letters, then the resulting text is "a".

Your task is to determine whether you can obtain the string t, if you type the string s and press "Backspace" instead of typing several (maybe zero) characters of s.

AlphaCode's solution follows.

Screenshot of AlphaCode output

AlphaCode’s code and explanation of its workings. Click to enlarge

AlphaCode did rather well at that task. The pre-print paper states it ranked in the top 54.3 per cent of entrants in the contests it … erm … contested.

Yup, that means it beat plenty of humans. Humans who may well code for a living.

The paper considers the broader impact on the world of code that codes, suggesting one potential risk is "advanced AI risks" as automated coding "could lead to systems that can recursively write and improve themselves, rapidly leading to more and more advanced systems."

Before we need to worry about that, the paper ponders if automated coding is bad news for human programmers.

The authors' conclusion is that developers should be OK because "writing code is only one portion of the job, and previous instances of partially automating programming (e.g. compilers and IDEs) have only moved programmers to higher levels of abstraction and opened up the field to more people."

More information about AlphaCode can be found in this pot and this GitHub repo.

Read them closely, dear reader. They might be the competition for your next job. ®


Other stories you might like

Biting the hand that feeds IT © 1998–2022