Is it possible to achieve a single view of the cloud? The more cloud services that a company uses, the more complex it would seem to get. Maybe you use a bit of AWS here, some Azure there, and some Rackspace somewhere else. That might be complex enough, but add in your own on-premise cloud solutions and it gets even muddier.
Managing all of that from a single pane of glass might be more difficult than it looks, if commentary from Gartner analysts is anything to go by.
Gartner's go-to cloud computing analyst Linda Leong told an AWS event last year that clients frequently come to her and ask how they can build a single pane of glass to monitor all their clouds infrastructures. She advised them to stop trying.
A paper that she co-authored last year said that the tools to enable true single-pane management are not mature enough, adding that there are significant differences in cloud IaaS implementations.
Before you really get into this, and decide whether Leong is right, you need to really understand what we’re talking about when we describe a single pane of glass.
Paul Griffiths, technical director of the advanced technology group for Riverbed, reckons monitoring needs to capture everything from the server to the end user and back again. That would need to incorporate reporting and monitoring for the entire journey of interaction between end points.
“If your monitoring system is receiving constant data flows from multiple points in the customer network but less frequent and fewer points from within the public cloud then there is a potential imbalance in the overall clarity and accuracy of reports,” Griffiths said.
“If the ‘public cloud metrics’ can be as complete, then that goes a long way to ensuring your monitoring system is fit for purpose.”
Services like AWS CloudWatch certainly provide a ton of metrics, from networking through to application performance. However, there are a lot of these points to keep an eye on across a multi-cloud environment though - networks are just the start and you need to think about the health of servers, storage and applications, along with looking for potential security incidents, and maintenance cycles.
Getting a deep visibility into IT services in a single pane of glass has always been a challenge. Clive Longbottom, founder of Quocirca, told us: “It would be really nice to have, but when it has been tried before through the use of ‘frameworks’ such as Tivoli and Unicenter, it was found to be severely constraining and did not do what was required.”
When you get to the subject of performance, you see another problem: With public IaaS, you have to look at Internet performance as a whole. After all, that’s how customers are connected.
Mike Kane, senior project marketing manager at Dyn, thinks the concept of a single pane of glass might be a bit of a red herring. “The real question here is, why do you need a single pane of glass? I could have services running in 150 different data centres around the world, but can I actually do anything with a dashboard that shows me this?” he asked, calling these total-spectrum awareness dashboards little more than vanity projects.
“It’s impossible to view and use all of the data on display simultaneously,” he said. “What’s required is a sensible balanced approach to useable and useful dashboards that provide actionable insights to those that use them.”
Kane suggests that maybe better alerts are what’s important. Getting a back-end monitoring system that pings you when something’s awry and sends you to the right interface will be far more useful than a vast array of data in a single window, he argues. In short, there’s no point putting the entire haystack online because the needle will be just as invisible. Instead, why not just poke you with the needle when necessary?
An alternative point of view allows for single pane, but just changes the front end interface, allowing for different views. Subsets of smaller haystacks, if you like.
Longbottom recommends getting a common dataset, based around the same concept as a configuration management database (CMDB). “That CMDB may be a massive great data lake itself, or it may be disparate sets of data that are pulled together by the over-arching system. Then, the monitoring, management and action items that are needed can be layered over this as plug-ins,” he said.
The plug-ins can be used to create dedicated interfaces for the different specialists involved. The storage admin sees one view that excludes anything not related to storage. The same goes for server managers, database admins, mobile specialists and whichever other roles you’ve carved your organization into.
How do you get all of that back-end data? You can query some APIs directly. AWS’s Elastic Beanstalk PaaS has an enhanced health monitoring service introduced in August 2015 that lets you do this, and Amazon’s CloudWatch services lets you submit custom metrics created by your own applications via an API request.
There are cloud management tools that already do this querying for you. Fordway’s Cloud Monitoring as a Service product takes CloudWatch data and uses an AWS integration tool based on Microsoft Systems Centre to integrate that data with its own toolset. It offers similar services for Azure, Office 365, and others. It can also include agents into instances or services to provide more detailed analysis.
Whether you start with a systems management tool, a DCIM tool or some other large-scale system, you can build dashboards that help you to monitor all of your cloud resources. The questions on your list should include who you imagine will be looking at it, and what level of data they’re hoping to see. You may find yourself building the specific back-end integrations yourself, and then having to stay on top of them as both your own internal systems and the public cloud service you’re using evolves.
Consider also the level of service you’re hoping to monitor. Infrastructure is just one part of the puzzle. If you’re exploring platform-based cloud services (PaaS) and online SaaS applications, then you’ll need to fold these into the picture, too.
It’s tractable, but be prepared to devote considerable time and effort to it – or work with a third party that can bear the load.
So then, it's a mixed picture. Some claim to already offer that single pane approach while others back Leong; still more even question the need for a single pane. One thing is certain: The extent you wish to engage in a single-pane-of-glass exercise is entirely down to you. ®