The ghosts of Microsoft SQL Server past, present, and yet to come: The Reg chats to Azure Data man Rohan Kumar
WinFS, stability, and waking engineers
Interview Everyone has their favourite when diving into the tub of SQL Server - for some it's a chewy hard toffee that delights, for others it's a thin choc shell crammed with overperfumed fondant. Microsoft's Rohan Kumar says his preferred version is SQL Server 2008.
El Reg spoke to Kumar, corporate vice president for Azure Data, about its history, future, and just why that particular flavour of DB is dear to him.
A brief history of SQL Server
Microsoft's twist on enterprise data goes back to the 1980s and the development of OS/2 undertaken by the soon-to-be Windows giant and IBM. Even as the relationship famously soured, Microsoft was working on an enterprise database product to compete with the likes of Oracle.
The result, a variant of Sybase SQL Server for IBM OS/2, was released by Microsoft and Sybase in 1989 as SQL Server 1.0. 1990 saw version 1.1, also for OS/2, before version 4.2 and 4.21 (for the freshly released Windows NT 3.1) arrived in 1993.
Microsoft and Sybase parted ways in 1993 and while the Sybase story ends in 2010 with an acquisition by SAP, Microsoft continued to toil away at SQL Server. Its first post-Sybase release came in the form of SQL Server 6.0 in 1995, followed by the ANSI-compliant 6.5 in 1996, before a major rewrite brought forth 1998's SQL Server 7.0.
WinFS, SQL Server, and Bill Gates
Kumar joined Microsoft around this time. "I used to be a part of the file system team," he explained. "Bill Gates had this vision, back in the day, which is sort of laid out in multiple ways of trying to expand the management of a relational database to start including unstructured data.
"And so they were looking for talent across the company that would help, you know, work on the vision that Bill had, which is that databases should be managing a lot more than structured data, given the power that they had. And that led to this project called WinFS."
WinFS was demonstrated in 2003 and was intended to be a new generation of file storage before eventually being shelved in 2006.
"That's how I got into what used to be the SQL team, because the charter for WinFS was with the SQL Server team back then. We were trying to basically marry file systems and databases. And I was brought into that.
"If you look at file streams, the technology that we have in SQL Server, or FileTable, a lot of the index, search functionality that we have, all that came out of the WinFS project.
"And that's how, starting with the 2005, 2008 versions, we started getting into more unstructured data management, XML files and having this whole notion of having a namespace for file systems within the database."
Everybody has a favourite
While Kumar has spent nearly two decades involved with SQL, he still regards himself as "one of the newbies" and added that there are still quite a few engineers working on the project who date back to the early 1990s.
Although the version counter has continued to tick up (and an Azure version debuted in 2014), Kumar named SQL Server 2008 as his personal favourite release. "It's very close to my heart," he told us. "It is why I was brought into what used to be the SQL team. And then we did FileStream and followed that up with FileTable.
"Those remain very special to me because after that, I sort of stopped writing code..."
Kumar has overseen seismic shifts in SQL Server over the last decade, from a porting to Linux, for which he paid tribute to an early design decision to abstract things via the SQL OS layer (evolving into the Platform Abstraction Layer), to the creation of a version of the core SQL engine capable of running in less than 300 megabytes of memory on an Arm chip.
The former came as Microsoft embraced the open-source operating system in recent years, while the latter is an intriguing move. Aimed at devices on the edge, it may yet find its way into data centres as Arm hardware increasingly crops up in servers.
Kumar was cautious about Arm's encroachment into the data centre world: "I wouldn't say a trend," he said, "but definitely, it's an interesting point."
"We obviously have investments, you know, that may or may not become something that we ship. That all depends on the business need… but some of the hardest things that we had to do [to run SQL on Arm], we're already past that."
Staying stable in the cloud
The future of SQL Server is now tied to the cloud, hybrid, public or multi (although at least one more on-premises version is in the works). As well as competition from the usual structured storage suspects, developers have NoSQL options from Microsoft rivals, such as MongoDB or the company's own Cosmos DB. Kumar also observed that "one of the fastest growing things we see in the cloud is serverless usage of SQL DB."
With the move to the cloud come the occasional wobbles of Azure and its peers. "My single biggest thing that I focus almost on a daily basis, is the reliability, stability, and availability," said Kumar.
"Upwards of 50, 60 per cent of all our investments go towards reliability, availability.
"With SQL, any outage that sort of goes beyond a minute, 90 seconds, is flagged as an incident that we need to go take a look at [and] engineers get woken up."
As for how many times those engineers have had to be woken up, or how many incidents have happened, Kumar told us: "Two, three years ago, that was in the hundreds. Now it's... maybe single digits."
Looking ahead for SQL Server
The immediate future for SQL Server, according to Kumar, is in hyperscale and big data clusters. "You'll [also] see us do a lot more time series, streaming analytics happening within SQL Server at the edge."
Azure Synapse and Arc also feature large in the immediate future, and Kumar was keen to draw attention to the Predict function and the training of models in Synapse: "That's also like, in fact, one of the biggest reasons why SQL on the edge is very powerful, is because of that Predict function."
"People train the models deployed on SQL on the edge, and they just keep scoring as a stream comes in, a transaction comes in, look at that against the model and score."
"So that's that whole modern paradigm of AI within the databases is something we are investing in pretty heavily." ®