Microsoft preps Azure data lake flood gates for readiness

And yes, it’ll feature analytics as a service


Microsoft will later this year release its open-source friendly lake for big-data analytics in the cloud.

Azure Data Lake Store – announced at Build in April – will be released as a preview, Microsoft said Monday. The store will be built on Apache YARN for developers and data scientists, to analyse information, and will use Azure HDInsight – a managed service for Hadoop, Spark, Storm and Hbase.

HDInsight was developed by Microsoft working with Ubuntu-shop Canonical and Hadoop spinner Hortonworks, Microsoft and Canonical said. In preparation for the preview Microsoft Monday threw the doors open on managed Linux clusters.

Microsoft’s Visual Studio Tools, meanwhile, have been updated to build, debug and tune for Hive queries and Storm topologies running in HDInsight.

YARN is the HDFS-compatible Apache project that makes MapReduce more flexible because it decouples resource management and scheduling from data processing.

The idea is you can support a broader variety of applications.

Microsoft’s data lake – a phrase now with huge currency among the big-data analytics and infrastructure providers – will feature analytics as a service.

Data Lake Analytics will use Microsoft’s U-SQL language it claims “unifies the benefits of SQL with the express power of user code". You’ll use U-SQL to analyse data in Azure, Azure SQL Database, and Azure SQL Datawarehouse.

The firm billed Azure Data Lake Store as HDRS for the cloud, capable of chomping petabyte files and that would be “enterprise ready". ®

Broader topics


Other stories you might like

  • Despite 'key' partnership with AWS, Meta taps up Microsoft Azure for AI work
    Someone got Zuck'd

    Meta’s AI business unit set up shop in Microsoft Azure this week and announced a strategic partnership it says will advance PyTorch development on the public cloud.

    The deal [PDF] will see Mark Zuckerberg’s umbrella company deploy machine-learning workloads on thousands of Nvidia GPUs running in Azure. While a win for Microsoft, the partnership calls in to question just how strong Meta’s commitment to Amazon Web Services (AWS) really is.

    Back in those long-gone days of December, Meta named AWS as its “key long-term strategic cloud provider." As part of that, Meta promised that if it bought any companies that used AWS, it would continue to support their use of Amazon's cloud, rather than force them off into its own private datacenters. The pact also included a vow to expand Meta’s consumption of Amazon’s cloud-based compute, storage, database, and security services.

    Continue reading
  • Microsoft's identity services huddle under Entra umbrella
    Decentralized identity and knowing who needs what

    Microsoft has whipped out the rebranding team once more, and chosen the name "Entra" as a catch-all for the company's identity and access capabilities.

    Sadly nothing to do with the sometimes-missed Encarta, the encyclopaedia launched by Microsoft back in the 1990s and eventually killed off in the first decade of the twenty-first century, Entra is not so much about squinting at postage stamp-sized bits of video and more about Azure Active Directory (AAD) and Cloud Infrastructure Entitlement Management (CIEM).

    While AAD is a familiar friend (or foe) to administrators responsible for an estate utilizing Microsoft's cloud, CIEM is a recognition that multicloud is a thing and management of identity over a variety of infrastructures and services is needed. This chunk of Entra has been dubbed Microsoft Entra Permissions Management and is designed to automate the principle of least privilege as well as unifying cloud access policies over a user's cloud infrastructure.

    Continue reading
  • Scribble to app: Microsoft's Power Apps VP talks us through 'Express design'
    Recognizing your doodles and building some software – or that's the idea

    Interview "We implemented this as an 'AI+human' experience," said Ryan Cunningham, VP of Power Apps, "and not 'AI does it 100 percent for you' experience."

    He was speaking about Power Apps, Microsoft's graphical software for creating low code applications, which is previewing a new feature called "Express design" that recognizes your scribbles.

    The thinking is that as a human tells the tools what's what, the AI gets trained over time.

    Continue reading

Biting the hand that feeds IT © 1998–2022