This article is more than 1 year old
Croquet for Unity: Live, network-transparent 3D gaming... but it's so much more
This is that rare announcement that's far more significant than it sounds
Updated Which of these appeals more: a new, free, Javascript framework for writing network-enabled 3D games, complete with integrated physics modelling and spatial audio… or, a complete, mature, dynamic programming platform that can implement the metaverse?
Why not both?
The new announcement from Croquet Corporation at the Game Developers Conference, GDC, in San Francisco is a very rare thing to come across the Reg FOSS desk's virtual – er – desk in multiple ways. Although it's far removed from our usual stuff about Linux distributions, virtualization, and so on, this is much more important and could impact many more people. We hope you'll bear with us for a few paragraphs while we try to explain why, because we think it's a genuinely big deal.
On one hand, this is trendy stuff – 3D gaming, immersive VR, the metaverse, and so on. It's a new Javascript framework for network-transparent multi-device 3D apps. It can now talk to Unity, which we've previously called "the most popular cross-platform game engine favored by video game developers". Although it's mainly in Javascript, it also supports Rust compiled to the cross-platform WASM runtime. It's free, but the company has a revenue model. If it's not already apparent, Croquet's press release, and its website, are both highly buzzword-compliant.
Croquet bills itself as "the OS for the Open Metaverse" and that is not an exaggeration: this could be one of most significant bits of software technology FOSS desk has seen so far this century. However, to get mainstream developers interested, the company is lightly disguising it, and marketing it as a tool for making it substantially easier to write cross-platform multiplayer games that work seamlessly over the internet, on desktop, laptop, tablet and smartphone. It is all that, but it's a lot more.
One thing Croquet isn't is a startup. Although this version of its eponymous product has been completely rewritten in vanilla Javascript, its first version was about 20 years ago, built on a transformative technology that was first released in 1985 and went open source in 1996. Despite the buzzwords, the tech is mature, with roots in designs that are older than Microsoft Windows. That also means that it's tiny, efficient, fast, lightweight, and in places, surprisingly simple. It's also got direct links to our story last week about the Xerox Alto and the three ways it influenced modern computers.
Croquet is a clever combination of some cutting-edge tech with some very powerful ideas that have been refined over decades. This is in part because Croquet was originally built by some exceptionally smart people, most of whom today act as its advisors. Among them are Alan Kay, who largely invented the graphical user interface; Dan Ingalls, who implemented the original Smalltalk and several important later derivatives, and on the side invented pop-up menus; David P Reed, who invented the UDP protocol; Mark P McCahill, who invented the URL and Gopher; and VR pioneers Julian Lombardi and Avi Bar-Zeev. We've written about Bar-Zeev before, but if you don't recognize the name, he built the original Google Earth (and much of Second Life), while Lombardi invented ViOS, probably the earliest attempt to commercialize the metaverse concept.
Croquet Corporation's founder and CTO is David A Smith, who wrote arguably the first ever first-person 3D shoot-'em-up The Colony, released in 1988 – four years before Wolfenstein 3D in 1992. He also did the 3D visualization tool for the movie The Abyss, which led to the first commercial 3D visualization app, Virtus Walkthrough. Alongside James Cameron, he's built projects with Tom Clancy, Michael Crichton, and worked for companies including Lockheed-Martin.
So it's really exceptionally safe to say that these folks really do know their stuff, because they invented it.
Croquet was originally implemented in Squeak. Derived from Apple Smalltalk-80 [PDF], released in 1985, Squeak is a Smalltalk whose VM is written in Smalltalk itself, by the same Dan Ingalls who implemented the Alto version at Xerox PARC.
There have been many implementations of Smalltalk, as well as later languages that it inspired. Notable among these is another Xerox PARC project, Self. Initially a dialect of Smalltalk, at Sun Microsystems this grew into its own thing: language rich, complete, and fast enough for a version of Smalltalk to be implemented in it. Self's just-in-time compiler turned into the Hotspot JIT in what is now Oracle's Java JVM. The most significant offspring of Self by far, though, is a version of it with curly-brackets syntax, whose creator described it as "a quickie love-child of C and Self. That is, of course, JavaScript.
We were surprised by the glowing terms in which Smith described it to The Reg:
JavaScript is an extraordinarily good language. I mean, it actually has three different languages it inherited its capabilities from. One was Self, which was itself built on top of Smalltalk. The second was obviously C, for the syntax, but the third was actually Lisp, for some of the things like closures and other capabilities. So, an extraordinarily powerful language. Most people don't understand that: they think it's a toy language, but in fact, it's probably one of the best languages out there. The other thing is, it's had billions of dollars of investment to make it very, very, very fast and very efficient, and it runs everywhere.
This is the point at which Croquet switching from Smalltalk to JavaScript suddenly made sense to us.
When this vulture first saw demos of Croquet in 2004–2005, it looked like a fun experiment in extending Smalltalk's two-dimensional, overlapping-window model into an 3D VR-style environment, using OpenGL to make it fast. This resulted in a translucent world, with windows that floated in mid-air. It looked to us a little like everything was underwater. It's intended for collaboration: each instance can talk to other instances over the network, synchronized via a lighweight protocol called TeaTime invented by the aformentioned Dr Reed. The Croquet environment could even embed other apps, such as a Mozilla-based web browser.
Right on the front page as it was in 2004, it gave the then relatively modest hardware requirements:
Our tests indicate that Croquet performs well even with Pentium II-450s and motherboards/components of that era as long as they have 256MB of RAM and a recent video card.
In the FAQ, they narrowed the latter down:
An nVidia GeForce 2 or better will work quite well and is reasonably inexpensive.
20 years ago, Croquet seemed quite large and complex. Now, specifications like these are trivial, and it runs on a far more heavily optimized, as well as ubiquitous, VM. Even the cheapest budget smartphone exceeds them by far, has hardware 3D rendering and fast, always-on communications.
Part of the problem, though, was persuading people to try it. Smalltalk is as much an operating system as it is a language, and most Smalltalk implementations run inside a self-contained VM, which is one of the things that has hindered its uptake. The main way to deploy a Smalltalk application is to ship an image of a VM with all of its components in place and ready to go. Smalltalk apps didn't look native on most OSes, and it's not trivial to keep Smalltalk code in a VCS such as Git, because Smalltalk apps are full of complex in-memory objects, not plain text.
But today, the most widespread programming language is a direct descendant of Smalltalk, which runs in a heavily optimized VM that's bundled with every browser on every computer and every fondleslab.
Smith told us:
So, it was pretty clear JavaScript would be a really great place to build. The browser would be the ideal place to build this next step, next-gen OS.
Of the current version, he said:
The system was greatly influenced by the earlier Smalltalk incarnation, but was a ground-up rewrite, based on many things we learned in earlier versions. Particularly important was the separation of the system into a model/view architecture.
The tooling is rich, mature, widespread and very well known. There are nearly 30 million developers in the world, and some 65 percent of them use JavaScript. It's a perfect fit. One of the company's demo videos shows VS Code editing a snippet of the code that controls the world the presenter is in, while the code is also visible in a live window in that world.
This degree of integration was unimaginable in the old Smalltalk version. And now, this tool can talk to Unity, too.
All Croquet virtual environments are independent and can be protected, but they run in sync with all other instances of that world thanks to TeaTime messages sent to internet servers called "reflectors". Like the protocol, these are light enough to run on edge devices. Croquet Corp hosts its own, and charges for their use, so while the code is free, there is a revenue model.
No matter how many devices they are running on, new instances can join at any point and sync within seconds. There is no separate client-side code talking to a complex server with state to be transmitted. Indeed, you don't need to write network-access code at all. As the documentation says (and the emphasis is in the original):
This synchronization is largely invisible to the developer. Creating a Croquet application does not require the developer to write any server-side or networking code.
For comparison, think of live migration of VMs from one host to another: a snapshot of its state is saved, and while the VM is loading into memory on the new host, the state is sent across from the old host, so that the VM can pick up where it left off within seconds.
Here, rather than VMs, each Croquet instance loads from a URL, executes entirely within that browser, and the reflector just keeps them in step.
Developers don't have to use Unity. There are a variety of other tools, such as Worldcore, which is available direct from NPM.
The company has a simple, two-minute explainer on its homepage, with a short video demo at the end, as well as some snazzy demo videos.
But this vulture felt that it was the examples that started to make it more impressive, and more fun, though. Run any one of the programs embedded in the page – we liked the Fountain under 3D. A fountain of 3D shapes starts; click your mouse, and you can throw new ones into the stream.
Each applet has a little hamburger menu at the bottom. Click this, and one of the options is to display a QR code. Scan the code with a handheld, and in a few seconds, it joins that instance, showing the same, live, content. Tap the phone's screen and the object you threw instantly appears on all other connected devices, collisions and ricochets and all. Other demos are more practical, involving live shared whiteboards, or collaborative code editing.
Although each instance is sandboxed within the browser, different virtual environments can communicate, via "portals": live, realtime windows showing the contents of other instances, and if granted permission, users can step through the portal into that other instance. While embedding a browser in Smalltalk was impressive stuff in 2004, now that kind of trick is easy. Croquet environments can contain multiple live web views, and Croquet code can also fetch data from external sources and render it live within the virtual environment. It makes it very easy to, for instance, create a virtual exhibition space that users can enter, explore, and interact with, and any changes propagate instantly to all other instances – all in free Javascript code, without a client, on any device.
We were initially skeptical when we learned that one of the 21st century showcase applications for Smalltalk had been converted to Javascript, but after playing with a few demos, we're convinced. Croquet and JavaScript were made for one another.
Smalltalk shaped the UI of every modern end-user computer, even though most of them don't use Smalltalk. By the early 1980s, Smalltalk made developing windowing GUI apps literally child's play: primary school kids could easily assemble working, interactive programs, without ever knowing what arcane concepts from 1960s OSes like "files" or "directories" were.
The multiple multi-billion-dollar corporations of the modern software industry have spent decades transplanting these UI technologies on top of multiuser, multitasking OSes built on 1970s technologies such as C and Unix. After 40 years of epic development efforts, building GUI apps is now hugely complex and difficult, the domain of paid professionals maintaining vast multi-million-line codebases of extremely fragile code, much of it in non-memory-safe, compiled languages such as C and C++.
But at the same time that this was happening, Smalltalk was evolving into a new language that is now inside almost every computer. Quietly, mostly unnoticed, the people from Xerox PARC and the people who built TCP/IP and the Internet, were moving on to things far more ambitious.
Moving beyond overlapping rectangular windows and into internet-scale immersive-3D environments. Built, not from static, fragile, compiled languages, but live, late-bound languages, code which can be edited, on the fly, in the tool itself while you're running it. Collaboratively, over the internet. And yet this is almost a peripheral attribute: this is a side-note, a nice extra for the people building it.
The applications for it are things like turning the web from a set of flat pages that link to one another, into a live collaborative virtual environment… with a physics model thrown in for free… with two-way gateways to and from other instances, and existing web pages and web apps… which is also a rich programming environment as well, but works with existing local tools too… and can also be used for building games, media players and other end-user apps… where putting together an application comparable to Zoom or Microsoft Teams is not a major tech undertaking involving hefty cloud servers, but a minor, almost trivial tech demo.
- Linux Mint 21.2 and Cinnamon 5.8 desktop take shape
- Official: Lomiri desktop now runs on Debian
- Fresh version of Xfce, the oldest Linux desktop of them all, revealed in Xubuntu builds
- By order of Canonical: Official Ubuntu flavors must stop including Flatpak by default
Where "serverless" doesn't mean "it's on someone else's server but you don't need to know where", it actually means "there are no servers".
And it's free, you can embed it into existing web pages, and it works with your existing infrastructure.
We suggest taking a look, or better still, a try. Never mind screenshots, videos cannot do this justice. Run some of the code examples, scan the barcode on a phone, and watch them interact. It's probably the single most impressive tech the author has seen since the first GUI computers in the early 1980s, and it's free to use. ®
Updated to add:
Smith has been in touch since the publication of this article to clarify Croquet's positioning on open source: "All of our frameworks are open source. The kernel of the system – the virtual machine and the reflector – is currently not open source though.
"This is mainly because of our concern about handing our business over to Amazon. We plan to open source the entire system once we are fully deployed. In fact, the fully decentralized system we will be making available next year will need to be open source."
He added that while the org has pricing for its web-based products (here), it has "not established pricing for Croquet for Unity yet – this will be set once we have the commercial release in the next few months, but will be very competitive."