Cloudera serves up 'secret sauce' for unified data management
CTO: Engineers spend half their time on this, customers need educating on the ingredient list
Cloudera has become the latest company to offer up a set of tools to unify data management, with its SDX framework, which it describes as the platform's "secret sauce".
Businesses are increasingly looking to ensure portability of data across a range of environments and flexibility in the ways they use that data.
The demand is driving companies to make sure they're offering something that clearly addresses that need, and for Hadoop-flinger Cloudera this is SDX, or the "shared data framework".
However, speaking to The Register at the Strata Data Conference in New York, Amr Awadallah, founder and chief technology officer, said that for Cloudera this has been more of a "framing exercise".
"The capabilities already existed," he said. "SDX is about framing that as our secret sauce that makes the Cloudera platform shine compared to our competitors."
Awadallah said that the move was partly to educate customers – saying many weren't aware of the work the company does on making sure data management is unified across the platform.
"Forty, if not 50, per cent of engineering time is spent on how to provide this unified experience. We've pulled almost 30 projects from open source, and are working to provide the same authorisation and way of doing things across them," he said.
"In the past we didn't communicate well that this was a key asset. But the customers need to have the flexibility and interoperability... otherwise they could just get the elements open source."
The SDX framework supports multiple public cloud, private cloud and on-prem, and allows customers to use and analyse their data in a range of ways, for instance Spark for machine learning, Impala for analytics.
"Companies want portability – they want to move between on-premise and cloud, and between clouds. They don't want to be locked into Azure, Amazon or Google. They also want the flexibility to extract value out of data in different ways," Awadallah said.
"It's about having the same access controls, policies, compliance, metadata management across a mishmash of different computation and storage."
Security, governance and compliance are heavy focuses in Cloudera's framework – something that harks back to Cloudera's first customers being in the financial and government sectors.
The framework offers authorisation for people to access the system, fine grain access controls, encryption for highly sensitive materials and auditing for who accessed the information, where and when.
Another announcement from Cloudera at the conference, also aimed at making products and platform can be used by the biggest possible range of customers, was that its Platform-as-a-Service Altus is now available on Microsoft Azure.
Altus was launched with Amazon in June, and the addition of Azure offers much the same as the previous one. Awadallah said that Google Cloud was next on the list, expecting it to be available next year. ®