What developers really, really want: Document databases

Why MongoDB tops DB devs' wish-lists

Sponsored Developers used to suffer from Rolling Stones syndrome – they couldn't always get what they wanted, or what they needed, to do the job. In 2020, things are finally changing for the better. Today, their bosses are letting them choose tools that make it easier to roll out solutions in the cloud – and that is especially true with databases.

When you’re building an autonomous fleet of robotic forklift minions like Toyota Materials Handling Europe you need happy developers with the right tools.

Filip Dadgar, the company’s principal architect and IT manager, knew that developers burdened with the wrong databases wouldn't be able to handle the data that it had spent eight years collecting from 80,000 forklift trucks. He put their needs front and centre, giving them systems that they could work comfortably with.

"It should be easy for developers to use and understand the data store," he says.

Toyota joined a growing number of companies opting for document databases. It used MongoDB Atlas, the global cloud database service, to manage the millions of data points that it had collected over the years, securely stored in easily digestible and queryable JSON-like documents.

R-E-S-P-E-C-T

Dadgar epitomises a new kind of manager: one that understands the importance of happy developers to a business that thrives on code. Technology is driving business strategy more than ever according to McKinsey, with 47 per cent of the top performing organisations creating entirely new digital offerings compared to an overall average of 28 per cent. Winning and retaining developer talent in this environment is critical.

Analyst firm RedMonk articulated this change in its book, How Developers Conquered The World. It described an emerging economy where the value of developers was increasing so quickly that companies were moving mountains to acquire and retain them. It advised them to "optimise for developer joy".

For developers, joy means getting the tools they want – and they’re telling us very clearly that they want document databases. In Stack Overflow’s 2020 Developer Survey, MongoDB topped the list of databases on developers’ wish lists, with the largest percentage of respondents stipulating it as the one they most want to work with.

Developers' Document Delight

A document database ranked first because of the advantages that it offers developers. Relational databases store data in a brittle schema that is difficult to change after the fact. Developers must put a lot of thought into their data models before writing a single line of code when using relational structures, ensuring that they support as many use cases and data relationships as they think they might need. That makes it difficult to get a project off the ground quickly.

The document database changed all that. It’s one of several database types that fall under the so-called NoSQL umbrella, which abandoned complex, rigid tables in favour of flexible, nested data structures. A document model stores data in a JSON-style document that uses fields (think of these as columns in a relational table) and their values. Document databases have several advantages over RDBMS software.

Documents map to the objects in your code, so they are much more natural to work with. A document’s schema is dynamic so it's more flexible and can incorporate changes and represent complex data structures easily. These traits have led to JSON documents being near universal across the application stack.

Documents are also human-readable, so that developers can understand what they contain more readily. These features make document databases much more intuitive for developers, improving their productivity and removing friction from the development process. It also eliminates the need to reconstruct records with complex SQL queries or get into extra object-relational mapping.

A document database like MongoDB supports a range of index types so that developers can also optimise performance for their applications’ data access patterns (and they can swap these out if the application’s query change).

These attributes, and JSON's ubiquity, mean document databases let developers get started quickly and refine their applications on the fly, explains MongoDB’s director of market intelligence Dominic Wellington.

"Documents evolve seamlessly with your application. It's a lot more fluid, and you have the flexibility to continuously optimize or incorporate new data types,”," he says. “Combine these traits with a powerful query language and JSON’s ubiquity and developers get a very consistent, very intuitive experience that allows them to work with data however they need to."

Storing a wide range of data types and having a document structure that you can change over time is a huge advantage in applications like Toyota’s, where developers need to iterate and evolve new concepts as they go along.

"From its earliest days, one of the driving factors in adoption of MongoDB has been the document model," explains Stephen O’Grady, Principal Analyst and co-founder at Redmonk, and the author of the aforementioned book. "Accustomed to having to understand and rigidly define the schema for relational databases up front, the document model allowed developers the flexibility to engage and experiment with what they knew and adjust as they went along. This decreased the friction of iteration and improved the velocity overall."

Cloud-ready and proud

Another important consideration is cloud-readiness, explains Wellington. "Most relational databases predate the idea of the cloud," he says, whereas MongoDB was developed both on the cloud and to run there. With MongoDB developers can automatically distribute documents across multiple servers, making it easier to scale out document databases in the cloud.

Another advantage is the ease of deployment. Developers want to automate their development and delivery pipelines, and they know that cloud-based practices like DevOps are important (80 per cent of Stack Overflow's respondents said so). The technologies that underpin cloud automation like containers and orchestration are native to those products developed in the cloud.

Cloud-native operation was especially important to Auto Trader UK, which made a conscious decision to change its ways and listen to its developers. Mohsin Patel, the company's principal database engineer, explains what things used to look like and how they shifted. "If someone came to us and said 'I've got a new application and I need a database' we didn't really give them a choice," he recalls. "We said to them, 'here's your Oracle schema. Go use it."

He found that developers experimenting with document-based databases would push back. "They're coming to us as and saying, 'actually I want my database to sit on this because that's what I've tested against.'" He realised how important developers were to Auto Trader UK because the world was shifting under its feet. It had evolved from a magazine business to an online operation that supported 50 million page impressions a month, and to cope with that rapid change it needed happy, productive devs.

Auto Trader UK leaned in. It has grown its MongoDB databases in 2019 while the number of Oracle database clusters in its portfolio has fallen.

It also adopted a cloud-native database, moving from its own data centre to MongoDB Atlas while modernising its application base. This enabled developers to deploy applications in minutes and then scale them quickly while giving them a consistent development pipeline. "We want to give our developers a seamless experience of our platform," explained Patel. "Every developer, whether you're working with Java or with Python, should get the same experience."

Now that everyone has the warm fuzzies, business managers are more likely to let developers choose the tools they need, confident in the knowledge that they'll deliver what the business wants. And developers? Well, they’re finally going to get NoSQL satisfaction.

Sponsored by MongoDB