Boffins stole our 3D files – and gave them all to Facebook's AI eggheads, claims Lithuanian biz

Planner5D wants lots of Zuck's bucks as compensation

Facebook used a purloined database of 3D objects for its AI projects, according to a Lithuanian company that spent years and millions of dollars compiling the dataset.

UAB Planner5D has sued the antisocial media giant and Princeton University in the US, claiming the posh college illegally downloaded more than a million digitized objects and scenes from Planner5D's website.

Facebook is named in the lawsuit because it used Princeton's allegedly purloined data in its AI efforts.

The objects compiled by Planner5D covered typical household and office stuff and layouts. According to court documents [PDF] filed in California this week, Facebook used the database to train up its robots to identify typical three-dimensional spaces.

Planner 5D cannot be accused of underestimating what it believes is the importance of its database of doors, windows, desks, chairs and lamps in different settings. "Whoever first masters this technology will forever change humankind’s relationship with machines," the lawsuit argued, somewhat improbably.

It said the database can be used to help elderly people from falling over, and in warehouses to help people spot and pack goods – which is all well and good but hardly a fundamental change in how humanity functions.

Regardless, Planner5D reckons the use of its database was not authorized and "strikes at the heart of Planner 5D’s business objective." In other words, it expected to be able to sell access to its vast database to help organizations like Facebook teach AI systems, yet the web giant simply grabbed the data and used it without asking or paying.

Smartest tools in the kit

The actual alleged theft was apparently carried out by Princeton researchers, the lawsuit stated. We're told the researchers wrote code to scrape Planner5D's website and ended up downloading more than 5GB of data. Planner5D controls access to its database by requiring users to access it through its own proprietary software: something it claims the researchers knowingly bypassed.

We said no scrapers!

The website's terms of service are also clear, the lawsuit claims, and specifically states that people are not authorized to use "any ‘deep-link,’ ‘page-scrape,’ ‘robot,’ ‘spider’ or other automatic device, program, algorithm, or methodology which perform similar functions to access, acquire, copy or monitor any proton of the Planner5D project…"

There is no doubt that the researchers grabbed the database, the company says, because they then wrote about how valuable the Planner5D database has been in their research papers. Somewhat amazingly, the researchers then posted and promoted their own copy of the Planner5D's database on their own servers for others to download, the lawsuit alleges.

If you visit the Planner5D website now it actively makes you agree to these terms and conditions before you are allowed to enter the website.

Zuck's bucks

Where does Facebook come into this? The researchers were part of a program funded by Facebook called the "Scene Understanding and Modeling (SUMO) Challenge" and Facebook is now using the models that were created as a result, the lawsuit claims.

"Like Princeton, Facebook is exploiting the Planner 5D dataset for the same purpose Planner 5D has set for itself: to train artificial intelligence applications to recognize 3D interior scenes…. Worse, Facebook explicitly secured from SUMO Challenge participants the right to commercialize the fruits of their work."

It goes on: "In return for their chance at cash prizes and the opportunity to present their winning submissions, SUMO contestants granted Facebook a 'perpetual, royalty-free, no-cost license and right to use and otherwise exploit' the submitted materials" – which, to be fair, does sound like a very Facebook thing to do.

According to Planner 5D, the terms of service on its website are clear and by grabbing and using it both Princeton and Facebook has violated them and so broken copyright infringement law as well as trade secrets laws.

The company wants damages – so far unspecified – plus costs and attorney fees. It also wants a block on Facebook's use of the database and for it to destroy any copies it has of the data.

We have asked Facebook for comment and will update this story if it gets back to us. The issue is due in court for an initial scheduling session on September 9. ®