A report from the FinOps Foundation has found that many organisations pay more than they need for cloud services, with challenges including getting engineers to control costs, and too much use of on-demand pricing.
The nonprofit is focused on cloud financial management and has members across more than 1,200 companies. It was formed in February 2019, partly as an offshoot of customer advisory board meetings at Cloudability, a FinOps company later acquired by Apptio. The FinOps Foundation became part of the Linux Foundation in June 2020.
The State of FinOps Report is based on around 800 responses from FinOps practitioners, with nearly half the respondents representing companies with 10,000 or more employees, and 70 per cent having more than 1,000 employees. As you would expect, they represent heavy cloud users. 31.1 per cent said they use public cloud for all their IT infrastructure, and a further 39 per cent described themselves as public-cloud first. 60 per cent of those surveyed spend more than $1m on cloud annually. The respondents also self-rated their FinOps maturity as beginner, intermediate, or expert – or, as the report put it, crawlers, walkers, or runners. Only 15 per cent said gthey regard themselves as expert.
The black bar represents FinOps beginners, mauve intermediates, and grey experts. Beginners pay the most
Why is cloud spending difficult to control? The biggest challenge, according to those surveyed, is getting engineers – concerned primarily with making stuff work – to take action to minimise costs. This was true at every level of maturity. The second biggest issue is allocating shared costs, and the third is getting accurate forecasting, which is difficult because of the complexity of cloud pricing.
How do you get started with FinOps? It begins with the native costing tools offered by the big public clouds, such as AWS Cost Explorer, Azure Cost Management, and GCP Cost Tools, but dedicated cloud cost management tools are popular too, at least among this crowd. Cloudability and VMware's CloudHealth attract about 25 per cent usage each, while others go with generic tools such as Splunk and Datadog. 26 per cent of respondents have developed homegrown tools – indicating, perhaps, that this is far from a mature market. When it comes to forecasting cloud spend, the top tool is the humble spreadsheet, most likely Excel.
Euro cloud slingers fight for niches on their own doorstep as AWS, Microsoft and Google inhale market shareREAD MORE
One intriguing aspect of the report is the correlation between FinOps maturity and cloud provider choices. Most organisations use more than one cloud provider. While Azure is second to AWS overall, as we would expect, "our advanced respondents indicate much more AWS and GCP usage," according to the report, while newcomers generally use AWS, Azure, and on-premises. The FinOps Foundation did not offer any interpretation of these figures. AWS and GCP arguably have more in common with each other than either do with Azure. Amazon and Google were founded as cloud companies, whereas Microsoft adapted itself to the cloud era, makes more use of Windows rather than Linux, and makes a greater play of hybrid technology.
There are two ways to reduce cloud spend. One is to use fewer resources (taming those pesky engineers), and the other is to pay less for them, using the deals on offer for reserved instances, committed spending (savings plans), spot or preemptible virtual machines, or even comparing across clouds and getting the best value on offer.
The report analyses this according to FinOps maturity. The "crawlers" use more than 60 per cent on-demand pricing, the report suggested, which means they pay the maximum, while "runners" use only 33 per cent. An AWS savings plan, for example, gives up to 72 per cent reduction for EC2 (virtual machines), Fargate (containers), and Lambda (serverless). Even the "runners" use 40 per cent on-demand pricing. There is always a role for on-demand, for unpredictable workloads or experiments and tests, but experts in FinOps aim to cover most cloud expenditure under a discounted plan.
In DevOps (from which FinOps got its name) automation is recognised as key to efficiency, reliability, and agility. The same may be true for FinOps. 12 per cent of those surveyed have managed to automate use of savings plans and reserved instances, and a similar number claim to have automated "rightsizing", the business of purchasing the right specification of infrastructure for each cloud-based solution – which all too often has elements of guesswork. More common are tools that send automatic recommendations to teams, used by 31 per cent, and those that enforce tagging hygiene.
Tagging is important because it means resource costs can be allocated to a specific project or department. Darek Gajewski, an infrastructure analyst at online genealogy network Ancestry, speaking at a FinOps Foundation event, recommended tagging enforcement, automatically shutting down untagged resources, "but also warned that this approach isn't something to take lightly."
Such practices are rare, though. In this report, "nearly half of our respondents (and almost 70 per cent of walk stagers) have little or no automation, and only 18 per cent actually automate infrastructure changes," we were told. Cost management automation is not yet mature. "Implementing cloud cost optimization automation is no small feat. Once it gets started, FinOps practitioners tend to reap many benefits."
Given the large potential savings on offer, FinOps looks like a rewarding area to investigate. The foundation seems to recognise that for many it is uncharted territory. It identified three top needs: a roadmap of what FinOps capabilities an organisation needs, benchmarking data to understand what can be achieved, and tool surveys to explain what is available and what the tools do.
The foundation promised further work on these goals, adding that "there's a lot more to come from deeper analysis of this data" – so watch this space. ®