Can't get no satisfaction in cyber recovery? Minimum viability is the answer

You might find you can get what you need. At least with a little planning

Sponsored feature Who could have anticipated that Mick Jagger knew so much about business continuity? "You can't always get what you want" was a light-hearted commentary on disappointment and acceptance, but today it's turning out to be a guiding principle for modern businesses facing a cyber attack.

Thanks to a concept called minimum viability, you might find you can get what you need. At least with a little planning.

Time is (not) on my side

Minimum viability has caught on as a concept out of necessity, says Sam Curcuruto, director of product marketing at cyber resilience software company Commvault. Back in the early days of disaster recovery when the Stones penned their song, events affecting businesses were mostly physical. Floods, fires, and power outages were the things to watch out for. Companies backed up their data and did their best to restore it quickly to another site.

You could trust the backup data in conventional disaster recovery, but things have changed in the last two decades. Physical events are certainly still issues, perhaps now more than ever, but there's another kind of flood: a rising tide of cyber attacks.

The first thing that companies must be able to rely on is the integrity of their infrastructure and their data, but as cyber threats mount, trust is becoming increasingly rare.

"The first thing that is lost in any cyber attack is trust," Curcuruto says. "When you're attacked, you can no longer trust the infrastructure or the data. Attackers can find ways to implement back doors that give them perpetual access to your environment."

Companies are under pressure to recover operations quickly, but a lack of confidence in what they're recovering slows that process down. Incident response to cyber attacks involves complex processes such as containment, forensic analysis to see what has been affected, and eradication of the threat from the infrastructure.

It's hard to predict recovery time objectives (RTOs) and recovery point objectives (RPOs) for that kind of response. Like a builder who doesn't know how long a renovation will take or what's involved until they see what's behind that old wall, it's hard to meet strict service level agreements when some jerk in Russia just pwned your entire system.

The results aren't good. Unless an organization can restore itself quickly and reliably, its business continuity will suffer. That has its own effect on compliance and reputation.

Shine a light

We must shine a light on these extra pressures facing companies in cyber crisis, says Curcuruto, and rethink recovery. The natural inclination has always been to restore everything at once, but that has to change. In a modern cyber event, limited time, resources, and trust in your own systems means that companies must narrow their focus, first restoring what's essential and known to be clean to operate safely and effectively.

From a cross-sector perspective, Curcuruto defines three tiers of system that organizations should look at when considering their minimum viability plan:

Tier one: Critical systems. These are the systems that you can't do without. They include identity and authentication (you can't use a system you can't log into), along with communication and collaboration tools (especially important in an era of hybrid work).

Also on the VIP list are the sector-specific systems that are crucial to your company's core mission. If you're a hospital facing a ransomware attack, your main focus is hopefully on patient outcomes. If your electronic medical record systems or patient scheduling systems go down, people could die. If you're a bank, maybe your mobile banking system is the most important thing as a customer-facing system, along with any apps supporting its basic functionality.

Tier two: Important but non-critical systems. These are nice-to-have but not must-have systems, and they often include non-customer-facing tools. Things like internal HR tools will often fall under this category, as might reporting systems.

In our hospital scenario, if it takes a little longer to recover your billing systems at the expense of patient care tools, then that's likely workable.

Sometimes definitions of tier one and tier two systems can be nuanced, Curcuruto warns. For example in manufacturing, you'd think that getting production back online would be the priority. However, that might depend on the type of product you're making, the inventory you have, and the urgency of getting it delivered. Depending on those considerations, getting shipping and delivery logistics operational might be first on your to-do list.

Tier three: Supporting systems. These are systems that typically have the least direct impact on short-term continuity or external-facing operations, such as archives, rarely used applications, and training platforms (unless you're a training company, of course).

Get on my cloud

To support all of this, IT has to consider the effect of architecture on the minimum viability process. One key area to examine is cloud-readiness, given the importance of the cloud in modern infrastructure. Legacy systems that can't move to the cloud limit what's possible. That might mean taking a stab at an RTO in that particular case, and if it's not short enough (meaning it's a higher-priority system) then considering other options like refactoring for the cloud.

This all sounds doable, but only if IT and business leaders speak the same language, Curcuruto says. In the past, when companies tried to recover everything, the whole affair was a more technical pursuit. With minimum viability in mind, business departments must have a greater say.

"It involves a cross section of the organization, largely focused on the operations and also the executive levels to be able to determine what minimum viability looks like," he says. "It requires an assessment of all of the systems that are in existence, and then it involves the prioritization of the things that actually power the business."

That might also involve more politics, he warns. Each business team might decide that their process is more important than another's, so prepare for some infighting. That in turn requires its own kind of soft skills and negotiation tactics.

Ideally, a company will be able to create a decision-making framework to help speed this along and avoid getting caught up in ad hoc politics. These are often facilitated externally, says Curcuruto, by involving experts from consulting and services teams who can be objective when helping prioritize.

Start me up

In addition to decision frameworks, Commvault also handles a lot of the technical needs of the minimum viability process, including preparation and backing up of systems, apps, and clouds in line with the customer's needs. 3-2-1 backup models are common, where you maintain three total copies of your data: two locally but on different media, and one off-site or air-gapped from your live environments.

Commvault also uses compliance locks and versioning to create immutable backups. "So, if you think about 3-2-1, at least one of those backups is immutable or air-gapped from your production environments. We can support all of those different levels of backup and protection," Curcuruto says.

The company also protects information using threat and anomaly scanning during backup to detect mass changes or other indicators of compromise. Integration with threat detection vendors like CrowdStrike helps to enhance this detection capability, as signals from those other tools can trigger actions within Commvault, such as locking down access to prevent further compromise.

Mother's little helper

Commvault's mission is to improve cyber readiness for all of an organization’s workloads, on-premises, hybrid, in the cloud, and everywhere in between. By providing this broad coverage, they can help organizations achieve their minimum viability, no matter where their apps and data live.

"We can automatically map and discover applications that exist within your clouds, along with their different configurations, their dependencies and the data that runs in them, so that we can then rebuild that, either in your own cloud of choosing or within a clean room," he says.

That clean room is a cloud-based virtual recovery environment that you can create on demand, also using automated tools, to rebuild infrastructure and data. The company also supports using gold master images for operating systems and applications - meaning that you get a trusted software stack without relying on potentially compromised backups.

As part of that recovery process for cloud data in Amazon S3, the company includes a service it calls Backtrack. This rolls back to a specific version of data at a file or database level without having to restore a full backup. That's useful when trying to get minimum viable operations up and running quickly. Managing restoration processes like these when recovering from a cyber attack is difficult enough, but it can be even more complex when you're triaging systems for recovery based on a minimum viability tier.

"You should try to use the opportunity to orchestrate and automate what you can in the recovery process, and that includes things like recovery-as-code," Curcuruto says. Using automations, paired with things like clean rooms and immutable backups, helps alleviate a lot of the complexity that comes with stressful situations like a cyber attack. These capabilities also make it easier and faster to execute cyber recoveries.

If there's one mission that Commvault has, it's to stop you having your 19th nervous breakdown (or even your first) when managing cyber recovery. To enable that, Curcuruto urges companies not to wait until there's a crisis before factoring minimum viability into their recovery process.

“It’s not enough to simply have a lot of these capabilities at your disposal. You also need to test your recovery plans in the good times to prepare for the bad,” Curcuruto concludes. That way, you can go into a real cyber recovery scenario cool, calm, and collected.

Learn more at Commvault

Sponsored by Commvault