This article is more than 1 year old
Microsoft teases Azure Data Explorer connector for picking its Synapse analytics service's brains
What do you mean you're not on board the Big Data bus?
Microsoft has rolled out an Azure Data Explorer connector preview for Azure Synapse.
The Azure Synapse Analytics service is all about bringing together enterprise data warehousing and Big Data analytics for the purposes of wrangling and reporting. It first popped up towards the end of last year (although it was once known as Azure SQL Data Warehouse) and Rohan Kumar, corporate veep for Azure Data, was keen to talk up the integration of the service with the likes of Apache Spark.
The new preview is an extension of the Azure Data Explorer Spark connector, natively integrated into Azure Synapse Apache Spark pools.
It's a little surprising that it has taken this long to emerge. Azure Data Explorer is all about peering into streams of data emitted by software then storing and analysing it. Microsoft has cited use cases such as ingesting telemetry from IoT kit and performing ad hoc queries.
Combining those near-real-time analytics with what might be lurking in Big Data has required the odd or shim or two in the past. The arrival of the connector should see data cached and indexed in Azure Data Explorer, and made available to Azure Synapse for querying purposes. Alternatively, data lurking in the Data Lake store connected to Azure Synapse Workspace can be shunted to Azure Data Explorer for more analysis or machine learning.
Exported data can also be queried with Synapse SQL serverless pool.
The connector is a handy tool in Microsoft's efforts to persuade customers that Azure Synapse Analytics is a better bet than the likes of Google's BigQuery and AWS RedShift.
While the Windows giant will trot out all manner of benchmarks to back up its case (as will its competitors), it is the implementation of connectors and their ilk that can make the difference for those tasked with making the Big Data dream actually work. ®