Updated Big clouds need a tightly integrated stack, and a small number of hardware components that can be upgraded and swapped out en masse. This is precisely what Verizon lacked in its first cloud, and what it is gaining with a new service.
The new Verizon public cloud was announced by the company on Thursday, and will see it be cost competitive with the major clouds operated by Google, Microsoft, and Amazon. To achieve this, the company has taken a completely new approach to how it builds its cloud gear.
Initially, the cloud will offer an object store, a block store, and a compute service with finer levels of selection for memory sizes on instances than Amazon. However, everything will be sold on demand in one-hour increments, versus Google's 10-minute increments, and it won't be possible to buy cut-price reserved instances like those available on Amazon.
"We have a global scale cloud platform we've been running for about five years," Verizon Terremark CTO John Considine told The Register. "We talked with customers and partners and really looked at the fundamentals of the platform."
Where the cloud was previously based on the ESX VMware hypervisor and used tools such as vCenter to perform data-center automation, the new cloud uses the Xen hypervisor instead of VMware with a Verizon-designed custom provisioning engine. In a break from the current fashion, Verizon is not using OpenStack in any way, shape, or form, Considine confirmed.
"What we've learnt is the [OpenStack] orchestration layers and tech are doing some very interesting things, but not solving the problems associated with a global-scale platform," he said.
"The problems with orchestration and cost and complexity aren't necessarily into how to virtualize that stuff. Where we run into a lot of problems is when you talk about independent lifecycles for all of those components. We really had to build what we consider an integrated system to make it work. We control everything from the manufacture of the disk drives down to the firmware on the storage cards; we find it has to be tightly integrated."
The decision could also be due to Verizon's long-running strategic relationship with Citrix on a bevy of technologies, paired with the freedom afforded from the Xen hypervisor being chucked into the Linux foundation back in April.
Breaking with tradition further, Verizon has moved away from the type of FrankenStack espoused by Oracle, and into a unified world more akin to the hulking clouds operated by Google, Amazon, and Microsoft. Where the previous cloud was built on HP and Cisco blade servers, plus a mishmash of EMC, NetApp, and HP storage, along with Cisco for networking and additional tools for load balancing, the new cloud uses a much more streamlined stack relying on a single vendor for compute and storage, and a single vendor (Arista) for networking.
"We don't use the big iron components anymore. It's different to the 1U commodity server approach – we've gone with fabric-based computing," Considine explained.
If not a standard 1U or 2U pizza box, then what? Well, we established that the system is a dense cluster of processors with integrated networking and storage and is significantly larger than a standard server. And it isn't an HP Moonshot, nor is it a custom job from an Asian manufacturer like Quanta or Wiwynn.
The company has developed an extremely flat networking architecture that means any VM in a set of 250,000 is a maximum of one hop away from any other, which should allow Verizon to quickly spin up clusters even in well populated facilities.
Initially, the cloud will be run out of a single data center in Virginia, but the company hopes to expand it over 2013 and 2014 to cover some seven facilities around the world, including one in London. At launch, the cloud will be the same size as Verizon Terremark's current cloud service, Considine said.
Though the company is not using OpenStack for any of its internal technology, it will provide API compatibility with the service, along with full compatibility for Apache CloudStack and – of course – Amazon Web Services's omnipresent APIs.
"As we've looked at what the true opportunity of cloud is, it is potentially a humongous business," Considine said. ®
AMD has revealed that the Verizon cloud is built on top of Arista switches and an undisclosed number of SeaMicro SM15000 server appliances, running a mix of AMD Opteron and Intel Sandy Bridge-era Xeons.