Waferscale startup says it can stitch chips together with light
Interconnect tech promises 96TB/s die-to-die communications
Hot Chips As chipmakers look to scale compute to ever-greater heights, they’re increasingly turning to waferscale compute architectures to circumvent bandwidth and latency bottlenecks.
Cerebras’ dinner-plate sized WSE-2 AI accelerator and Tesla’s Dojo training tile are just two examples of how this tech is being used to scale well beyond the limits of a single or even handfuls of dies.
However, the approach remains complex to implement and is still confined by the latency and bandwidth limitations of electrical interconnects. This is exactly what a startup called Lightmatter is trying to solve with its Passage silicon photonics technology, detailed at the Hot Chips conference this month.
“Arrays of electrically interconnected chiplets suffer fundamentally from issues, including concatenating power consumption,” Nicholas Harris, founder and CEO of Lightmatter, said during the virtual event.
He explained that as the number of chiplets increase, chipmakers are bound to run into bandwidth and latency challenges, which limits the practicality of this approach. These limitations have already driven switch silicon vendors like Broadcom to explore co-packaged optical technology, with their Humboldt series of ASICs.
While effective for a single chip, Harris argues the relatively large cross-section of an optical fiber makes this kind of co-packaged optical interconnect impractical for dense chip-to-chip communications.
Riding the waveguides
Instead of using fiber optics to stitch together multiple chips, Passage uses nano-photonic wave guides. These are channels that can carry photons at extremely low-loss and at very high bandwidth.
“Passage is diced from a 300mm Silicon Photonics wafer that includes lasers, optical modulators, photo detectors, and transistors all side-by-side integrated in the platform,” Harris explained.
Passage essentially functions as an optical communication layer on which a variety of customer dies, including ASICs, CPUs, memory, or other accelerator dies can be placed and interconnected.
“Because Passage has integrated lasers and transistors, the co-packaged chips don’t have to deal with any of the complexity of the transmit, receive, or circuit switching photonics elements,” Harris said. “Each Passage tile can house an array of heterogeneous chips. For example, a tile might contain two different types of ASICs and maybe two HBM stacks.”
The idea here is that customers no longer have to worry about developing complex interconnect fabrics, like Intel’s EMIB or AMD’s Infinity Fabric, on their own. And because Passage is built on a silicon photonics wafer, it can be resized to accommodate up to 48 full-reticle dies.
“Passage is like a chocolate bar; you snap off as many squares as you need,” Harris said.
Nano-photonic waveguides also have advantages over traditional fiber optic interconnects, namely, they’re a whole lot smaller.
Lightmatter says it can fit 40 waveguides in the space of a single optical fiber. This, they claim, allows them to provide 96TB/s of bandwidth to each die at less than 2ns of latency. And, according to Harris, because this is achieved using optical circuit switching, traffic from any one die is at most a single hop away from another.
Communication off package is achieved via fiber arrays integrated at Passage’s edges. Each fiber attach provides up to 16TB/s of bandwidth, which can be used to interconnect multiple Passage tiles.
- Intel shows how chiplets will form Meteor Lake CPUs
- Tesla wants to take machine learning silicon to the Dojo
- Will optics ever replace copper interconnects? We asked this silicon photonics startup
- Intel set to squeeze the flops out of Ponte Vecchio GPU
Electrical communication is also supported using through silicon vias (TSVs), which also deliver power to the dies and support PCIe and CXL, Harris added.
Lightmatter sees several opportunities for this technology, including the disaggregation of memory and compute as well as dynamic allocation of resources in a composable infrastructure-like fashion.
Harris described one scenario in which multiple Passage tiles — each equipped with various compliments of memory, CPUs, and accelerators — could be interconnected using the fiber attach points. In another scenario, Harris described using the optical switching capabilities to effectively air gap resources used by different tenants.
“Within a millisecond, the entire platform can be reallocated across the userset,” he said.
Passage is far from the only company looking to silicon photonics to address the challenges associated with pushing ever-higher bandwidths farther without sacrificing on latency.
Ayar Labs is another startup exploring the tech for high-speed connectivity. The company has developed a chiplet that takes electrical signals from chips and converts them into a high-bandwidth optical signal.
And similar to Lightmatter’s Passage, Ayar’s TeraPHY is intended to be packaged alongside compute tiles from other chipmakers using open standards like Universal Chiplet Interconnect Express.
Established chipmakers, like Intel, are also actively exploring the use of silicon photonics-based interconnect tech.
Last year, Intel opened a research center to study the use of integrated photonics for datacenter applications. And, in June, Intel demoed an eight-wavelength laser array that’s integrated on a silicon wafer. ®