Open Compute 2013 Struggling server processor and chipset maker Advanced Micro Devices is getting some friendly help from the Open Compute Project, the open-source hardware effort started by Facebook in April 2011.
Or, conversely, AMD is being friendly with the Open Compute Project to try to boost sales of its latest Opteron 6300s now that the chip maker has worked with motherboard and server makers to get the "Roadrunner" two-socket mobo to market.
Last May, both AMD and Intel created custom motherboards for the bespoke Open Compute (OCP) V2 server chassis that Facebook's hardware engineers designed for use in the social network's North Carolina data center. They also previewed two other server mobos that would be aimed at high-frequency trading, storage array, and other more intense workloads.
The OCP V2 chassis crammed two half-width two-socket x86 servers and its power supply and disks into a 1.5U-high vanity-free chassis. (Meaning, it has no ornamentation or bezels or anything not absolutely necessary.)
The Intel Xeon E5 motherboard that slides into the box is code-named "Windmill" and the Opteron 6200 and 6300 mobo is called "Watermark," just so you can keep the names straight. In conjunction with the OCP V2 server, Facebook also said it was abandoning 19-inch wide standard compute racks with its Open Rack design, which allows for five 3.5-inch disks to be laid out in a row across the front of a server. (You can only do four in a standard 19-inch machine.)
The Roadrunner motherboard cooked up by AMD, which is technically known as the AMD Open 3.0 motherboard spec now that it is out, is not restricted to this 21-inch OCP V2 chassis and Open Rack design, and will still be available for 19-inch wide spaces. The Roadrunner board is 16 inches deep and 16.5 inches wide and can be used in standard 1U, 2U, or 3U server enclosures, Suresh Gopalakrishnan, vice-president and general manager of AMD's server business unit, told El Reg. There is no reason it can't be used in a 1.5U custom box that is a bit wider like the one that Facebook has designed for its own use.
The Roadrunner two-socket, Open Compute motherboard designed by AMD and friends (click to enlarge)
As you can see, the motherboard has two big notches cut out of it and power connectors that are soldered onto the ends of those notches. This way, the power supplies and motherboards can be packed tightly, leaving more room in the front of the board for memory.
The Roadrunner board is based on AMD's own "Fiorano" SR5670/SP5100 chipset. The Opteron 6300 processors have four channels of DDR3 main memory per socket and three DIMMs per channel for a maximum of a dozen DIMMs per socket. The board has six SATA (which are to the left of the power connector on the right side of the power supply notch on the motherboard above).
The board has two PCI-Express 2.0 x16 slots and two for PCI-Express 2.0 x8, with all four being physically x16 slots. The board also has a PCI-Express 2.0 x8 "mini-mezzanine" slot for plugging in 10 Gigabit Ethernet controllers if you need more than the two Broadcom 5720 Gigabit Ethernet ports that are welded to the Roadrunner board. Any mezzanine card that can plug into an x8 slot is supported, but the preferred one will probably be the ConnectX-2 cards from Mellanox Technologies, which can do Ethernet or InfiniBand links.
The mobo has a Broadcom 5725 management port and links into the Open Machine Management tools that the OCP has been trying to foster. It also has two USB ports.
Targeted configurations of the AMD Roadrunner systems
Under normal circumstances, motherboard makers would vary the number of memory slots on a mobo, but Gopalakrishnan said "customers are tired of that" and they don't want to commit to a specific maximum memory configuration before they buy. They want to be able to expand the memory as they see fit without having to swap out mobos or whole systems. So rather than make three different boards with three different memory configurations, the Roadrunner board has three suggested configurations aimed at three different workloads.
For cloud infrastructure, the motherboard is put into a 2U chassis and uses all the memory slots and has them running at 1.6GHz. The machine is equipped with a few disk controllers to augment the six SATA ports on the board, and a total of 25 2.5-inch SAS or SATA drives for storage. The power supplies and fans are powerful enough to allow for the use of the Opteron 6300 Special Edition parts, which burn 140W of juice.
For high performance computing (meaning supercomputing and high-freaky trading), the Roadrunner board slides into a 1U chassis and only puts one memory stick on each memory channel, for a total of eight sticks running at 1.6GHz and a goal of pushing that up to 1.87GHz soon. This machine uses the six SATA ports and has six SATA drives as well as ten 2.5-inch SAS or SATA drives and can also use the Opteron SE parts provided the fans and power supplies are chosen carefully. It has room for two low-profile or one regular PCI slot, and it is likely that a 10GE mezz card will be used, too.
AMD and its Roadrunner friends also suggest that the mobo would make an excellent foundation for a storage array, for which you would socket in some relatively modestly powered Opteron 6300 processors, fill up the memory slots, and cram 35 disk drives into a 3U chassis that would allow the use of all four PCI slots with full-height cards. Again, depending on the connectivity needs for the storage array, you could also use the 10GE mezz card, but in a lot of cases – think Hadoop Distributed File System – you could probably get by with Gigabit Ethernet.
It is interesting that none of these scenarios employ 3.5-inch disks, which are the preferred drives for Hadoop big data munching because they are the cheapest and fattest drives on the market.
A 1U rack server from Quanta using the Roadrunner Opteron motherboard
At the moment, motherboard makers Tyan and Quanta Computer are making variants of the Roadrunner mobo, and Penguin Computing and Avnet have committed to peddling boxes based on them. Gopalakrishnan was mum on whether mobo and whitebox server maker Super Micro would create its own variant of the Roadrunner board and make and sell systems based on it, but if they take off in the market, the odds certainly favor it.
Gopalakrishnan did not say if Facebook has itself used the Roadrunner boards in any of its systems, but he would confirm that Fidelity Investments has a few evaluation systems that it is using to test its code, as do a number of other unnamed financial services firms.
One more thing. Just because AMD worked with Broadcom, Mellanox, AMI, Nuvoton, Facebook, Tyan, and Quanta to come up with the Roadrunner design and did a lot of the testing and validation work to make sure the board works well, that does not mean that AMD will be making and selling Roadrunner motherboards nor systems that use them.
"We are not getting into the motherboard business," said Gopalakrishnan. Of course, AMD is in the server business directly thanks to its ownership of microserver and interconnect maker SeaMicro, but it has no plans to build whitebox Roadrunner machines. ®
