Oh no, you're thinking, yet another cookie pop-up. Well, sorry, it's the law. We measure how many people read us, and ensure you see relevant ads, by storing cookies on your device. If you're cool with that, hit “Accept all Cookies”. For more info and to customize your settings, hit “Customize Settings”.

Review and manage your consent

Here's an overview of our use of cookies, similar technologies and how to manage them. You can also change your choices at any time, by hitting the “Your Consent Options” link on the site's footer.

Manage Cookie Preferences
  • These cookies are strictly necessary so that you can navigate the site as normal and use all features. Without these cookies we cannot provide you with the service that you expect.

  • These cookies are used to make advertising messages more relevant to you. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests.

  • These cookies collect information in aggregate form to help us understand how our websites are being used. They allow us to count visits and traffic sources so that we can measure and improve the performance of our sites. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance.

See also our Cookie policy and Privacy policy.

This article is more than 1 year old

What do microservices do to data stores? Netflix is built on them and had no idea!

So it built a now-open-source benchmark tool for Cassandra and Elasticsearch

Microservices are the new black for developers, but even one of their world's biggest and most prominent users – Netflix – has said they're a bugger to manage.

“As Netflix runs thousands of microservices, we are not always aware of the traffic that bundled microservices may generate on our backend systems,” write the company's Vinay Chella, Ioannis Papapanagiotou and Kunal Kundaje.

“Understanding the performance implications of new microservices on our backend systems was also a difficult task.”

Which rather re-enforces the notion that perhaps the container ecosystem is a mite immature.

But we digress.

To get on top of its microservices, Netflix decided it needed “a framework that could assist us in determining the behavior of our data store systems under various workloads, maintenance operations and instance types.

“We wanted to be mindful of provisioning our clusters, scaling them either horizontally (by adding nodes) or vertically (by upgrading the instance types), and operating under different workloads and conditions, such as node failures, network partitions, etc.”

So Netflix built it, called it NdBench and open-sourced it. Which is why that link back there goes to GitHub, where you can get the code for your own footling pleasure.

What does it do? For now it can test Cassandra (Thrift and CQL), Dynomite (Redis), and Elasticsearch, the three data stores Netflix employs. It can simulate workloads, measure performance under different operating systems, identify memory leaks, thrash CPUs to see how they react and much, much more.

Netflix says it's used the tool for “the last few years,” so if you decide to download you're getting a reasonably well-tested tool. The streamo-supremo nonetheless says it is “looking forward to receiving feedback, ideas, and contributions from the open source community.” ®

 

Similar topics

TIP US OFF

Send us news


Other stories you might like