Software as a service (SaaS) can be a great cost saver for companies willing to abandon their own hardware and software, but what happens if productivity leaves the building too?
Giving control of your business applications to someone else can also mean losing control of performance.
When staff complain to the help desk that pages in a browser-based CRM application are taking 10 seconds each to load, and they have customers on the phone, who do you call? More to the point, how do you stop the problem arising in the first place?
Application performance management systems that use locally installed software agents to check on things like server and network load are useful when you are running systems on your own premises, but how do you monitor performance when all of the equipment is running on someone else’s turf?
On your marks
“There are application monitoring solutions available today that allow the end-to-end response time of a transaction to be measured,” says Tony Lock, programme director at Freeform Dynamics.
“This gives the organisation a means to calibrate the service quality of application performance without having to use any data supplied by the SaaS vendor.”
Some systems approach this customer experience management problem by monitoring performance from a specific URL.
TechOut offers a monitoring solution for web applications that is operated on a SaaS basis. It uses servers in different regions to perform various tests on a destination site, such as GETS and POSTS with variable parameters, and provides a report on all objects and frames in the page.
CA’s APM Cloud Monitor enables companies to monitor the state of SaaS applications and reports back on performance.
Blame the internet
It is difficult, however, for customer experience monitoring solutions to distinguish between the SaaS provider’s servers and the network. It would be easy for an unresponsive cloud-based application hosting service to simply shrug and point to the internet.
Networks can indeed be a bottleneck, warns Steve Marsh, EMEA product manager at cloud migration firm Metalogix. Such bottlenecks may be particularly noticeable when uploading or downloading large data objects, such as in a Sharepoint SaaS implementation.
“The main thing they are going to notice is uploading and downloading files from Sharepoint,” Marsh says.
“That’s something we need to play with. That would be the first measure. Do we have acceptable upload and download times for our files?”
The issue lies not just with the provider’s external network connection, but with the customer’s own WAN and LAN setup. Get it wrong and you could see a considerable slowdown.
“That’s when users turn around and say that the new online service is slower,“ Marsh warns.
A handy Ping or Traceroute command could soon put disputes about network problems on the customer’s side to rest. But how can you measure performance inside the SaaS provider’s infrastructure?
“With difficulty,” says Clive Longbottom, service director at analyst Quocirca. “You need an agentless probe that can get past their firewalls and not be seen as an intruder.”
Ideally, says Tal Mikaelovich, technical director at application performance management firm Precise, users would have more control over their third-party cloud environments.
“Cloud companies would like to bring more business but they know that they need to give some access to the servers, otherwise it’s not going to work,” he says.
Maybe. While you may be able to install software of your choosing in some platform-as-a-service scenarios, most SaaS providers are unlikely to let customers anywhere near their servers with third-party software.
Some providers are building performance reporting into their own system and service offering so they can challenge claims that they are underperforming, says Longbottom.
It is important to note the level of reporting here. Rather than send a report to customers only when services are entirely down, a reporting structure should indicate when response times dip below a certain threshold.
Examine the contract carefully for weasel clauses
That raises the question, what do you put in a contract? Service level agreements (SLAs) for SaaS implementations should offer more than just guaranteed uptime. Response times should figure in the equation.
“As in any contract, it is essential that customers specify upfront their service quality requirements, in whatever terms and metrics they need," says Lock.
“Get agreement on how these will be monitored, measured and reported, as well as identifying SLA remediation and modification, potentially over long timeframes."
Longbottom advises customers to examine the contract carefully for “weasel clauses" that allow the provider to shirk responsibility by simply pointing to a part of the infrastructure outside its control.
“Also, use the provider to advise on performance trending," he says. “It's no use having an SLA and then just waiting for it to be breached."
Customers must be able to work with the provider to prevent breaches from happening in the first place by analysing workloads against a baseline.
There are other ways to improve the performance of a SaaS solution, says Marsh. In storage-heavy SaaS deployments, such as SharePoint, it is possible in some hosted configurations to separate the storage from the application logic.
“We are looking at vendors now that have chargeback models based on how much is in Sharepoint," he says. “You can remove up to 95 per cent of your content from SQL, including binary large objects, and put that into alternative storage tiers."
These tiers could be tailored for high performance. “It could be that they are offering their fastest storage, versus cheaper storage," Marsh says.
For such actions to be effective, you need to understand your future capacity and latency requirements.
This leads us to provisioning. A responsible customer will work with the SaaS provider to plan for future growth. In an on-premise scenario supporting Sharepoint users, you need to build a server farm large enough to support the number of users and the amount of content that you plan to use.
“Balance that against how you might scale out in the cloud. It's a question of provisioning new sites and new quotas online," Marsh says.
“What is adoption going to be like? What is the online storage going to look like, and what will your agreement with the vendor look like over time compared with an internal solution?"
Just because your service provider is up there in the cloud doesn't mean you should treat it as an inexhaustible resource with no planning.
The needs of users must also be taken into account when deciding what aspects of your SaaS applications' performance to monitor. Response times for synchronous application sessions are one way to measure performance, but they are by no means the only one.
In some SaaS scenarios, file backup and restore might be an important performance indicator. If a user deletes a file by accident and asks for it to be restored, and the service provider says it will take three days to restore it, would that be acceptable?
Understanding what parameters of application performance you are trying to monitor is crucial.
The difficulty of piercing the cloud to get at a SaaS provider’s infrastructure may not be a big deal when you are dealing with one provider. But it could make the long-term goals for the cloud computing community harder.
We already know that some companies are eager to develop marketplaces of cloud-based applications that can be bolted together and used in a SaaS-based model.
Fujitsu is one such firm. The marketplace approach is part of its overall cloud strategy. But these environments open up several challenges, one of which is getting everyone to agree on a performance management standard.
If four companies contribute modules to a single SaaS application, persuading all of them to allow the same software to be installed on their servers, or even to adhere to the same performance reporting standards, might be an uphill struggle.
“Few will want to work in this way," Longbottom predicts.
Marsh has a sage piece of advice. “If you find yourself going down the cloud route and performance isn't as expected, then make sure you have a strategy for backing out of the cloud and coming back on premise," he says.
“You don't want to find yourself going down the SaaS path, stopping, and having to start from scratch." ®