Forget DeepFakes. This robo-Rembrandt with AI for brains is not bad at knocking off paintings

Fret not, artists, sellers and buyers ... it's far from perfect at this stage

28 Reg comments Got Tips?

AI-powered robo-painters are getting somewhat better at ripping off masterpieces, judging by the following fresh research.

A team of academics at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and Princeton University in America, and Chulalongkorn University in Thailand, have crafted a new system dubbed RePrint.

It’s split into two parts: a 3D printer that outputs layers of resin ink, and a trained neural network. When shown a photo of an oil painting, the system identifies the overall layout of the image, adjusts the lighting to compensate for whatever conditions the input picture was taken, and then predicts which colours are the right ones to mix and how to apply them to recreate the work using the printer.

The key thing here is to mechanically produce a painting based on a digital snap, mixing and daubing the inks as necessary, imperfections and all. It's, of course, possible to snub machine learning, and write some filter code to add a little distortion and fuzzing here and there, and determine the colour mix using algorithms, to ultimately drive a mechanical painting machine.

But where's the fun with that? And in this experiment, the software was working with photographs of paintings rather than close-up scans, which adds an extra dimension of difficulty that a neural network may be better at tackling than many lines of handcrafted filter code.

Here’s a comparison of an original painting of water lilies and the RePaint-generated knockoff:

Taken from Deep Multispectral Painting Reproduction via Multi-layer, Custom-Ink Printing

The image on the left is from an original painting by Azadeh Asadi, and on the right a copy produced by RePrint. Image credit: Shi et al ... Click to enlarge

“If you just reproduce the color of a painting as it looks in the gallery, it might look different in your home,” said Changil Kim, co-author of the paper and a postdoctoral fellow at MIT CSAIL. “Our system works under any lighting condition, and shows a far greater color reproduction capability than almost any other previous work.”

The described system is better than conventional 2D ink printers because it can pick from ten different colours instead of the normal four: cyan, magenta, yellow, and black. These inks are then stacked in very thin layers on top of each other to produce a more accurate colour, and spread to produce an accurate rendering of the painting. The neural network was trained with supervised learning to predict colour combinations with a training dataset containing more than 18,000 samples of ink stacks.

Image of woman holding mask of her own face

The eyes have it: 'DeepFakes' bogus AI-meddled videos outed by unblinking gaze


If you look closely at the results, however, RePrint misses tiny structures like the direction of brush strokes and the final work lacks the gloss of real paint. This imitation isn’t better than the real deal, yet. The researchers hope to improve their system by including more inks in their palette and changing the thickness of the different layered inks.

“The value of fine art has rapidly increased in recent years, so there’s an increased tendency for it to be locked up in warehouses away from the public eye,” said Michael Foshey, co-author of the paper and a mechanical engineer at CSAIL. “We’re building the technology to reverse this trend, and to create inexpensive and accurate reproductions that can be enjoyed by all.”

The researchers hope that their techniques can recreate lost or damaged fine art, and will present their work at SIGGRAPH Asia, a computer graphics conference, in Tokyo next week. ®


Biting the hand that feeds IT © 1998–2020