It started at Pixar. Now it's the Apple-backed 3D file format viewed as HTML of metaverse

Nvidia pushes 30-year-old description language for the internet of the future

An open-source file format adopted by Apple, and the origins of which can be traced back 30 years, is getting a fresh look as hype grows around the building of a metaverse without borders.

The Universal Scene Description, or USD, was described as the "HTML of 3D" by Richard Kerris, vice president of the Omniverse platform at Nvidia, during a press briefing ahead of its GPU Technology Conference this week.

Nvidia is backing the file format as a linchpin to build a collaborative metaverse on its Omniverse hardware and software platform, through which companies can build and render complex 3D worlds, AI models, and animated avatars.

The idea behind USD is to share and reuse hundreds of thousands of 3D assets across virtual-reality applications developed by multiple companies, panelists said during a discussion about the file format at the conference on Tuesday.

USD allows the sharing of 3D assets and rending virtual worlds in a collaborative way, and could bridge them together in a so-called metaverse. In theory, it could provide the hooks to bridge, say, Minecraft to Roblox.

The file format is "an important element because it allows for all these software products to take advantage of the virtual worlds we are talking about," Kerris said.

USD serves as a platform for sharing of virtual sets, animations, materials, and other 3D assets across applications or virtual worlds. It is also a tool for live collaborative scene building, taking into account position, orientation of objects, colors, layers, and more. USD has composition operators that combine all this data.

To render this, an engine reads the procedural description of how to stitch a scene together out of the shared assets, such as animations or models, and puts it together at runtime. Nvidia recently collaborated with Apple to describe rigid body physics simulations, too.

"It's actually the most comprehensive one I think that we've seen, because it allows you to robustly interchange not only geometry, but also shading materials, lighting, descriptions of how to render things," said Sebastian Grassia, project lead for Universal Scene Description at Pixar during the panel discussion.

Metaverse Inc

Pixar created the format to make sure it could share and reuse the hundreds of thousands of 3D assets for its movies. The format, which was opened up in 2016, allows the 3D data sets to be reused in live workspaces while keeping the original files intact.

"Pixar has been doing what we call scene description for about 30 years now, and started out very simply, and as the needs of our movies have gotten more and more complex, we iteratively produced more ambitious ways of describing models assets and needing to find more powerful ways to combine them, bring them together," Grassia said.

USD has given Pixar the flexibility for different departments to work independently, but also collaborate on lighting, color and layers before rendering a master scene. The file format is now being adapted to create and share 3D data sets for newer graphics pipelines and workloads like virtual reality.

The format is natively supported by Apple on its devices, and is being groomed to play a big role in the company's virtual reality future. USD is supported by Epic Games on its Unreal Engine, and the company is promoting use of the collaborative game development. Autodesk supports the format through its popular software suites. Nvidia now offers RTX rendering for USD files via its Omniverse platform.

The other 3D file formats include Alembic, which was originally announced in 2010 by Sony Pictures Imageworks and Lucasfilm Ltd, was used for films like Men in Black 3 and The Amazing Spider-Man.

While the USD file format promotes 3D visual collaborative sessions and protects source data, participants in the panel session held back on making USD the "HTML of 3D," as Nvidia has hyped it.

USD has promise, but whether metaverse builds up a data footprint around the format has yet to be seen, said Shawn Dunn, senior product manager at Epic Games. Much like HTML, the industry will have to work together to adopt a standard to render applications in the metaverse, he said.

"It's something that at least if everyone can contribute to it, suddenly, you have common language that as long as the other software packages can read it, you can visualize it in wherever you want," Dunn said.

Another idea was floated around USD possibly being the JPEG of 3D, but that was viewed as being far-reaching considering the growing popularity of glTF 3D file format, an open-sourced format backed by Khronos Group.

The USD file format may work for films, but fundamental challenges need to be address to make it feasible for the Internet and web browsers.

"I suggest USD needs to have a JavaScript library. Currently, everyone must manually create their own Python and C++ services to read the USD on the server side and then send the needed info back to the client side. If USD creates a library that can eliminate this work for the user, it will go a long way to ... allow use of USD in browser based applications," said Mohsen Rezaya, chief solutions architect at Siemens. ®