You looking for an AI project? You love Lego? Look no further than this Reg reader's machine-learning Lego sorter

All you need is tens of thousands of Lego bricks, a Raspberry Pi, and a laptop GPU

An engineer has built something that is sure to be the envy of any self-respecting Lego fan: an AI-powered Lego sorting machine.

Daniel West, a software engineer from Australia, built the device, which is also made out of Lego, of course, over two years. The impressive gizmo stands about 80 centimetres tall, contains over 10,000 individual Lego parts, and 15 motors to move pieces that need to be sorted along a series of conveyor belts. West claims it can process a bucket of jumbled up pieces, grouping 2,927 types of Lego into 18 different bins at a rate of one brick every two seconds. Not bad, eh?

You can watch the machine chugging away below.

Youtube Video

Here’s how it works: First, a giant fistful’s worth of random Lego parts are poured down a chute at the top of the machine. The multicoloured river of Lego is processed and shaken by a vibrational feeder to filter it into individual pieces that pass under a Raspberry Pi computer.

The computer contains a camera and snaps a series of pictures of the passing brick and sends them to a laptop running ResNet-50, a popular convolutional neural network, to categorize. The result is then sent back to the machine, opening the right combination of gates to allow the particular piece to fall into the right bucket.

Sorting Lego may be a trivial and tedious task for humans, but it’s actually pretty difficult for machine learning models. Lego parts come in thousands of distinct types, in multiple colours, and combined with the fact that they all look different when viewed at various angles, it’s no wonder that collecting the right combination of training datasets was the hardest part.

West told The Register this week he, at first, tried to generate fake simulations of the Lego bricks. He took 3D models of the pieces from LDraw Part Library, an open source program that allows enthusiasts to build Lego virtually, and rendered them in Blender, free animation software.

The 3D model bricks could then be simulated at different rotations and colors. These individual images were collected into a synthetic dataset containing over a whopping 25 million pictures. But to West’s frustration, his AI Lego sorter trained on the fake images failed to identify real parts.

“I came to a point where I almost gave up on the project entirely,” he said. “I spent months implementing a complex method to transform synthetic images into real-looking images, with little success.” So, he turned to another popular technique often used in robotics to help reduce the simulation-to-reality gap: domain randomisation.

World's first Universal Lego Sorting Machine?

Domain randomisation trains a model to recognize even more variations in data. Instead of just different rotations and colors of Lego bricks, the system learned to account for various lighting effects, texture, and noise. To boost the performance of the Lego sorter further, West also incorporated a smaller dataset containing snaps of real Lego pieces.

“The real dataset only contains parts from my personal collection which ends up being only 544 different types of parts,” he said. These pieces are more common types found among Lego collections, whereas the synthetic dataset features rarer types.

angry lego minifig man turns on anxious lego minifig man

How do we stop filling the oceans with Lego? By being a BaaS-tard, toy maker suggests


By training on both datasets, the camera on West’s Lego sorter should, in theory, be able to recognize pieces that it hasn’t actually seen before. In other words, the machine can categorize bricks in the synthetic dataset that isn’t in West’s personal collection. The performance varies, however, it’s 93 per cent accurate at identifying parts that are in the real dataset compared to 74 per cent for ones that are only in the synthetic dataset.

West trained his ResNet-50 using approximately 800 CPU cores rented on AWS over a day. The Lego sorter then runs on Nvidia’s GeForce RTX 2060 graphics card commonly found in gaming laptops. West believes that it would also work reasonably fast with weaker GPUs too. He is planning to release the code in the future.

It’s not the world’s first Lego sorter. West was inspired by other gizmos, some of which also used AI algorithms. He claimed, however, that it was the world's first Universal Lego Sorting Machine. “I call it 'Universal' because, thanks to the use of cutting-edge AI, it's capable of recognizing and sorting any Lego part that has ever been produced," he said.

“As a lifelong Lego fan, ever since I saw Akiyuki's original Lego sorting machine way back in 2011, I'd had the idea of building a similar machine in the back of my mind," West continued. "About three years ago, I became interested in computer vision.

"AI and CNN's were booming and I began studying them intensely. I also had a lot of experience with computer graphics as during this time, I had just released my video game 'Airscape: The Fall of Gravity'. I realized that I would be able to combine my skills in a unique way to finally realize my lifelong dream of building a real working Lego sorting machine, while at the same time gaining real-world experience in designing and deploying AI systems.” ®

Other stories you might like

  • North Korea pulled in $400m in cryptocurrency heists last year – report

    Plus: FIFA 22 players lose their identity and Texas gets phony QR codes

    In brief Thieves operating for the North Korean government made off with almost $400m in digicash last year in a concerted attack to steal and launder as much currency as they could.

    A report from blockchain biz Chainalysis found that attackers were going after investment houses and currency exchanges in a bid to purloin funds and send them back to the Glorious Leader's coffers. They then use mixing software to make masses of micropayments to new wallets, before consolidating them all again into a new account and moving the funds.

    Bitcoin used to be a top target but Ether is now the most stolen currency, say the researchers, accounting for 58 per cent of the funds filched. Bitcoin accounted for just 20 per cent, a fall of more than 50 per cent since 2019 - although part of the reason might be that they are now so valuable people are taking more care with them.

    Continue reading
  • Tesla Full Self-Driving videos prompt California's DMV to rethink policy on accidents

    Plus: AI systems can identify different chess players by their moves and more

    In brief California’s Department of Motor Vehicles said it’s “revisiting” its opinion of whether Tesla’s so-called Full Self-Driving feature needs more oversight after a series of videos demonstrate how the technology can be dangerous.

    “Recent software updates, videos showing dangerous use of that technology, open investigations by the National Highway Traffic Safety Administration, and the opinions of other experts in this space,” have made the DMV think twice about Tesla, according to a letter sent to California’s Senator Lena Gonzalez (D-Long Beach), chair of the Senate’s transportation committee, and first reported by the LA Times.

    Tesla isn’t required to report the number of crashes to California’s DMV unlike other self-driving car companies like Waymo or Cruise because it operates at lower levels of autonomy and requires human supervision. But that may change after videos like drivers having to take over to avoid accidentally swerving into pedestrians crossing the road or failing to detect a truck in the middle of the road continue circulating.

    Continue reading
  • Alien life on Super-Earth can survive longer than us due to long-lasting protection from cosmic rays

    Laser experiments show their magnetic fields shielding their surfaces from radiation last longer

    Life on Super-Earths may have more time to develop and evolve, thanks to their long-lasting magnetic fields protecting them against harmful cosmic rays, according to new research published in Science.

    Space is a hazardous environment. Streams of charged particles traveling at very close to the speed of light, ejected from stars and distant galaxies, bombard planets. The intense radiation can strip atmospheres and cause oceans on planetary surfaces to dry up over time, leaving them arid and incapable of supporting habitable life. Cosmic rays, however, are deflected away from Earth, however, since it’s shielded by its magnetic field.

    Now, a team of researchers led by the Lawrence Livermore National Laboratory (LLNL) believe that Super-Earths - planets that are more massive than Earth but less than Neptune - may have magnetic fields too. Their defensive bubbles, in fact, are estimated to stay intact for longer than the one around Earth, meaning life on their surfaces will have more time to develop and survive.

    Continue reading

Biting the hand that feeds IT © 1998–2022