Analyse this: Microsoft promises OLAP-OLTP 'Link' with new CosmosDB features

Azure SQL Data Warehouse

Ignite Microsoft has pushed out Azure Synapse Link for Cosmos DB to general availability in an effort to bring its transactional NoSQL database closer to the analytics workhorse data warehouse.

Teased in May last year, Microsoft said the link would comprise of two main components.

Firstly, a Cosmos DB would house a column-oriented analytical store within containers in addition to the existing row-oriented transactional store. "The analytical store is fully isolated from the transactional store such that queries over the analytical store have no impact on your transactional workloads," developers Ramnandan Krishnamurthy and Sri Chintala said in a blog post last year.

SQL - lake conceptual illustration

Databricks: Ugh, just look at that messy data lake environment. Squints. You know... we could sort that out with a sweet shot of SQL


Secondly, Azure Synapse Analytics would offer run-time support, that is, the "native integration" of the Azure Cosmos DB analytical store with the various analytics runtimes supported by Azure Synapse Analytics, such as Apache Spark and Synapse SQL serverless.

The idea of running analytics on live transactional data is not new by any means. In the relational database world, just a couple of examples include MariaDB's approach, involving holding columnar data in object storage within the database, and Oracle's Cloud Data Science Platform, which performs analytics directly on its business application databases.

Nonetheless, Synapase link for Azure CosmosDb - allowing punters to directly connect to their Azure Cosmos DB containers from Azure Synapse Analytics and access the analytical store with no separate connectors - was an important milestone for Microsoft, said Noel Yuhanna, veep and principal analyst at Forrester.

"We find a growing demand for near real-time analytics from many vertical industries, including financial services, retail, and healthcare. However, latency in data movement and processing from transactional systems often slows down this initiative."

Talking up the hookup between OLTP (transactional) and OLAP (analytical) systems, he added: "With Azure Synapse link for Azure CosmosDB it brings OLTP and OLAP closer together that will help organisations support real-time analytics with minimal effort," he said.

Microsoft's move to bring another Azure source into the Synapse environment would appeal to customers that are using CosmosDB for high-scale and highly distributed applications, while an additional MongoDB API makes CosmosDB more attractive to those familiar with document NoSQL database, said Doug Henschen, veep and principal analyst at Constellation Research.

"The integration between CosmosDB and Synapse will make it easier to make use of the data in CosmosDB in the data science, data engineering and SQL analytics contexts supported by Synapse," he said.

More Cosmos tweaks

Microsoft also announced the general availability of Mongo v4.0 server support in the Azure Cosmos DB API for Mongo DB service, designed to make it simpler for developers to use the database for error handling with multi-document transaction support and "retriable" writes.

Available in preview is Azure Cosmos DB Continuous Backup and Point-in-Time intended to recover and restore data from any point within the past 30 days. Cosmos DB role-based access control is also available in preview.

In the data warehouse side of things, Microsoft is launching Azure Synapse Pathway, a feature designed to simplify the move from a legacy or cloud data warehouse to Azure.

Constellation Research's Henschen said development promises "to bring popular third-party data sources, such as Teradata, Snowflake, IBM Netezza, AWS Redshift, SQL Server and Google BigQuery into Synapse."

Synapse is pitched against other cloud-native data warehouses including Redshift, BigQuery, and – with its $33bn IPO explosion bringing attention to the market – Snowflake.

Although it is early days for Synapse, "lots of companies and customers are at least kicking the tyres," Henschen said, although he added that it was "unclear how many of these deployments can be described as being in full production."

Synapse also marks itself apart with its use of Spark-powered data lake capabilities with a SQL-based warehouse environment running against shared and consistently secured and managed data, he said.

"It's not the same thing as the BigQuery, Snowflake and Redshift database services, but it represents competition in the sense that it's a cloud option for high-scale SQL warehousing and analytics.

"For now, it's early days for Synapse and I'm not seeing a lot of head-to-head competitive assessments yet. Synapse will clearly be an option that any Azure-centric customer will consider," Henschen noted.

Coming at the problem from the other direction, Databricks last year moved to bring SQL to the data lake environment, with the launch of its SQL Analytics to address BI in its Spark-based Delta Lake. ®

Similar topics

Other stories you might like

  • US won’t prosecute ‘good faith’ security researchers under CFAA
    Well, that clears things up? Maybe not.

    The US Justice Department has directed prosecutors not to charge "good-faith security researchers" with violating the Computer Fraud and Abuse Act (CFAA) if their reasons for hacking are ethical — things like bug hunting, responsible vulnerability disclosure, or above-board penetration testing.

    Good-faith, according to the policy [PDF], means using a computer "solely for purposes of good-faith testing, investigation, and/or correction of a security flaw or vulnerability."

    Additionally, this activity must be "carried out in a manner designed to avoid any harm to individuals or the public, and where the information derived from the activity is used primarily to promote the security or safety of the class of devices, machines, or online services to which the accessed computer belongs, or those who use such devices, machines, or online services."

    Continue reading
  • Intel plans immersion lab to chill its power-hungry chips
    AI chips are sucking down 600W+ and the solution could be to drown them.

    Intel this week unveiled a $700 million sustainability initiative to try innovative liquid and immersion cooling technologies to the datacenter.

    The project will see Intel construct a 200,000-square-foot "mega lab" approximately 20 miles west of Portland at its Hillsboro campus, where the chipmaker will qualify, test, and demo its expansive — and power hungry — datacenter portfolio using a variety of cooling tech.

    Alongside the lab, the x86 giant unveiled an open reference design for immersion cooling systems for its chips that is being developed by Intel Taiwan. The chip giant is hoping to bring other Taiwanese manufacturers into the fold and it'll then be rolled out globally.

    Continue reading
  • US recovers a record $15m from the 3ve ad-fraud crew
    Swiss banks cough up around half of the proceeds of crime

    The US government has recovered over $15 million in proceeds from the 3ve digital advertising fraud operation that cost businesses more than $29 million for ads that were never viewed.

    "This forfeiture is the largest international cybercrime recovery in the history of the Eastern District of New York," US Attorney Breon Peace said in a statement

    The action, Peace added, "sends a powerful message to those involved in cyber fraud that there are no boundaries to prosecuting these bad actors and locating their ill-gotten assets wherever they are in the world."

    Continue reading

Biting the hand that feeds IT © 1998–2022