Comment Microsoft's Azure storage offering suggests it is developing its own cloud storage operating system.
Azure was developed in a Windows Azure Group which was separate from the Windows and Servers Group. The two organisations have now been combined into a new Servers and Cloud Division (SCD) unit, headed up by a senior VP, Amitabh Srivastava. SCD itself is part of Bob Muglia's Servers and Tools Business unit.
The Azure cloud service, based on Microsoft's own data centres, is dependent on virtualised server instances, child partitions in Hyper-V-speak, using Windows Server 2008 R2 and Hyper-V Server 2008 R2. Users will access these in a public cloud across the Internet and set up virtual server applications. These applications will access storage that provides three kinds of data store: Blobs, tables and queues.
Blobs - binary large objects - are data and metadata in up to 50GB lumps. A Blob could be formatted as a single NTFS volume VHD (Virtual Hard Disk). For more fine-grained storage needs tables hold a simple hierarchy of entities with properties. These are not relational database tables, nor are they accessed by SQL. The idea is to provide for massive scale out across multiple storage boxes, with a table capable of storing several billions of entities constituting terabytes of data. Queues hold Azure infrastructure requests from one Azure application instance to request a service from another or send a message to another.
These instances, Hyper-V child partitions, are of two types: web role instances and worker role instances. THe web role instance communicates with users outside the cloud via incoming HTTP or HTTPS requests. Worker role instances are like Windows services or batch jobs. There is more information on the Azure infrastructure here (pdf) and here (pdf) and here.
The Azure storage offering does not map onto any existing storage supplier product offering. It's hard to see, for example, how NetApp storage or EMC storage could slot right in and offer Azure Blobs, tables and queues. Last year Michael Dell said of Azure: "This is a new platform which allows you to scale easily from 10 users to 10 million users without additional coding, and it all runs on Dell." Dell is the sole supplier of racked servers and storage hardware for Azure.
Dell's Forrest Norrod, VP and General Manager of its Data Center Solutions Division, says Dell is supplying customised server and storage iron to the Azure data centres. In a YouTube video he talks of server and storage node grids.
Dell's storage, the in-house EqualLogic arrays and the OEM'd CLARiON EMC gear, does not natively provide Blob, queue and table storage facilities, so what does?
Is Dell providing custom firmware for its EqualLogic arrays that turns them into highly scalable storage grids? That would lock Microsoft into Dell as a storage supplier. Both Dell and Microsoft emphasise open and industry standards talking of Azure so a customised Dell firmware idea looks untenable.
Microsoft could be layering its own server-hosted Azure storage management software on top of these Dell storage nodes with their controllers. Alternatively it could be writing its own storage array controller code and using relatively simple storage boxes underneath, JBODs (just bunches of disks).
Whichever option it has chosen, Microsoft is using and presenting an Azure storage array with the underlying physical controller and drive enclosure resources abstracted by, for want of a better name, an Azure Storage O/S. The ASOS functionality would include gridding the storage nodes together, indexing the Blobs and tables, and maintaining the queues. It must also protect the data and provide some form of load balancing accesses.
We have no information about whether Fibre Channel, SAS or SATA drives are used in the storage nodes, although Norrod does talk of the Azure Dell products being power- and performance-optimised, which suggests a SATA drive base.
There is an Azure fabric connecting servers and storage in the Azure cloud but this is quite separate from the underlying physical network connecting the servers and storage in an Azure data centre. The servers used in Microsoft's Azure data centre will be familar vanilla multi-core X86 running familar Windows Server software. Dell may well be supplying X86 embedded servers as the storage node controllers so combining the Azure and Server groups makes a lot of senses from this point of view.
If Microsoft decided to productise its Azure storage O/S then there is a possibility that a new Dell/Microsoft product could emerge, an Atmos-like product, which comes with the Azure storage O/S code on it. Now that would that upset a few storage applecarts. ®