What does it really take to be a leader when it comes to cloud databases?
Here’s what the AWS customer obsession means to you
Sponsored Feature Which is the best illustration of the central role AWS can play in the modern organization? It could be the fact that 50,000 customers headed to its re:Invent conference in Las Vegas in November to hear about its latest product updates, including a slew of database announcements. Or it could be that Gartner positioned AWS highest in execution and furthest in vision in its 2022 Magic Quadrant for Cloud DBMS.
In fact, both points support each other, says the company. AWS has become a leader in the cloud-native database market precisely because it pays such close attention to what customers need from their database infrastructure, and is obsessive about delivering it.
AWS manager of product marketing for database services Elizabeth Solomon says that when it comes to running databases in the cloud, it is clear: "The rapidly declining economics of public clouds and the 'pay per use' model have removed cost-related barriers to cloud adoption. So, we're past the economic tipping point."
In addition, the industry has surmounted "psychological barriers related to operational unknowns" particularly around data ownership, retention, availability, and security. This means that cloud databases have become the default choice for many customers, and part of the overall cloud-centric approach alongside IT disciplines and modern application design principles. Serverless has been a driver that makes cloud databases even more economically attractive because they allow customers to pay for only the database resources their workloads consume, says AWS. According to Solomon," six AWS database services are now serverless".
But if that explains why cloud databases are the first choice for those companies today, what are the specific reasons why AWS was rated so highly in the Gartner Cloud DBMS MQ last year?
There are lots of things behind it, says Solomon, but "our customer obsession is a top driver." AWS roadmaps are "highly influenced" by the challenges customers face, she continues and the company "views everything on the basis of positive customer impact – it's an important litmus test for our decisions."
AWS also benefits from being part of the broader Amazon.com organization and the retail side of the business specifically, she adds. "It's an enormous ecommerce platform with the most demanding requirements, and we've learned a lot about how to scale technology and how to scale operations." Database services such as Amazon DynamoDB and Amazon Aurora are core technologies that support Amazon.com.
No silver bullets
It's not down to any single killer app or "silver bullet" grabbing customers' imaginations though. AWS' entire approach to meeting customers' database needs is predicated on aligning with modern microservices-based application development architectures and delivering a portfolio of purpose-built databases designed for specific workloads. IoT workloads differ from OLTP which differ from gaming and so on.
"It is hard to find one database that can do all of these things and more, or find a single database that can do all of these things well at the same time," says Solomon. "More importantly, trying to consolidate all these service patterns in a single database reduces developer velocity as they have to coordinate service changes with each other."
That is why AWS range includes relational and NoSQL databases, which include several styles of purpose-built databases such as key-value, graph, in-memory, document and more. And it also provides the analytics platforms that enable customers to wring the most possible value and insight from their data.
"We support all workloads that help you act on your data," says Solomon. "From analytics with Amazon Redshift for SQL analytics and Amazon EMR for big data analytics using open source solutions, to machine learning with Amazon SageMaker, to business intelligence with Amazon QuickSight."
Of course, unifying disparate data sources is a challenge in itself. So, AWS is also focusing on what Solomon describes as a "zero ETL future so you can quickly and easily connect to and act on all your data, no matter where it lives."
The aim is to allow customers to catalog, connect, and govern all their data. "The various silos of data that exist within organizations make it difficult to find and combine the data that you need. Making this data discoverable and usable is key to solving this problem," says Solomon.
Customer journeys start here
Equally key is recognizing that each customer is in a unique place in their data modernization journey. Some simply want to continue using their existing databases but need new or additional data center infrastructure, in which case Solomon thinks Amazon RDS is an easy choice.
For others, removing the day-to-day drudgery of database admin is the starting point, she continues. "They take a forward-looking approach to their data strategy and adopt fully managed, purpose-built databases as part of a broader modernization effort."
Some organizations start with the rebalanced cost equation the cloud presents. This is particularly true for those who have used their legacy database for the last 10, 20, or 30 years, and are tired of paying the associated licensing fees. Solomon believes these customers are "breaking free" by moving to the cloud, more so if it involves an open-source or open-source compatible database.
At the same time, companies looking to take advantage of modern development methodologies find the cloud more amenable. "Application architects assume infrastructure transience, horizontal scaling and decomposition of applications into network connected microservices," says Solomon. "Infrastructure availability is measured in seconds. Agile development and DevOps are table stakes."
This is complemented by the ability to tap virtually unlimited capacity, and scale down while maintaining confidence in availability and security. The net result is that developers and DevOps teams can spend more time on schema design and application performance optimization.
That also gives them more scope to focus on new applications which are aligned with the broader digital transformation initiatives, says Solomon. "You can innovate faster and pay more attention to core initiatives that truly differentiate your organization. The move to managed databases is a very liberating shift."
The cloud is often seen as the default choice for startups, enterprises and everyone in between. As showcased at re:Invent 2022, US property insurance giant Liberty Mutual spoke of its use of Aurora Serverless v2 to underpin its Incentive Compensation application, allowing it to auto scale reliably.
Likewise, financial services company S&P Global - which is looking to build out machine learning models for sector wide asset classification - turned to Aurora Serverless v2 to scale its compute resources which supported all AWS Glue jobs, Amazon Lambda functions, and the ingestion of massive volumes of data.
Elsewhere, Disney+ used purpose-built AWS databases and AWS integration services to satisfy the needs of a broad set of data consumers with a dispersed organizational structure. As Solomon explains, "Their data team needed a scalable approach to data distribution but to still maintain a high degree of security and governance over the consumption of data."
Innovation trumps operation
These three customer stories mirrored a spate of re:Invent product announcements which were centered on three key themes.
"The democratization of advanced operational techniques removes big hurdles that get in the way of organizations that don't want to make massive investments in IT operations and specialized skills," Solomon says, which frees them up to focus on innovation, not operations.
Similarly, the launch of Amazon RDS Blue/Green Deployments is expected to increase availability by making database updates safer and much less disruptive by providing a managed staging environment. The intention is that updates can be carried out in less than a minute, with zero data loss.
"Performance and scale are important because of the deluge of data and types of data organizations are experiencing and will continue to experience," Solomon says. "For almost every organization this deluge of data is a clear and present day-to-day reality. Customers need reassurance that they have plenty of headroom and performance will not be a problem now or into the future."
Additional scalability and performance enhancements for Amazon DocumentDB and Amazon RDS are designed to help deliver that reassurance. They include the ability to horizontally scale writes through the sharding of data, while a series of internal optimizations have boosted the read and write performance of a single database instance.
Interoperability across different AWS services is important too, because it can improve productivity across development and operations teams and is perceived as a critical tool for building modern applications. Where this integration occurs is crucial, says Solomon. "Application developers often have to include this integration into their application code. If the integration is built in under the covers, then that's one big area developers don't need to worry about."
This is illustrated by the launch of point-to-point integration between Aurora and AWS RedShift analytics platform, effectively creating a zero ETL data pipeline that lets customers run analytics on their database data in near real time.
All of these new capabilities will help empower organizations that do not have advanced operational knowledge and expertise in house, says Solomon. "Security, durability, availability and reliability are no longer limited to early adopters with large budgets."
At the same time, she continues, AWS is continuing to invest in "security and governance, simplicity and ease of use, gleaning insights through powering our services with machine learning and more. Consistent delivery against all of these is at the root of the trust we have earned with customers".
As for what this means for the future and AWS' continued status as a leader in cloud databases? Solomon says, "We're totally energized by the possibilities."
Sponsored by AWS.