Information security (infosec) isn’t a game for amateurs. No one solution will do. Proper information security requires defence in depth: layers of technologies, techniques, best practices and incident response woven together into the tapestry of everyday operations.
Unfortunately, hiring professionals is no guarantee that information security will work, either. Perhaps the most relevant example is the United States' Department of Homeland Security's (DHS) completely useless 6-billion-dollar firewall.
The short version of the DHS debacle is that after more than a decade and $6bn of development it essentially does nothing. I have built Linux VMs using open source packages such as clamav, spamassassin, iptables, snort and squid that are more functional. I rebuild them every year. They take about a week to do properly for the latest version of the packages. I'll gladly build a set regularly for the DHS as a fully automatable virtual appliance for a lot less than $6B, if they're interested.
That, right there, is the problem. Everything the DHS was trying to accomplish had already been done – and done rather well – by Commercial Off The Shelf (COTS) software. With some minor work you could get open source packages to do most of the same thing. (Being honest, the COTS software is generally better and easier to use than the open source stuff for this use case.)
For $6bn dollars the DHS should have been able to weave together a masterpiece of COTS software packages with custom signatures and maybe combined it with a little bit of homegrown middleware or some open source glue to create an information security defence screen more capable and adaptive than anything anyone in the entire world had been able to build.
$6bn should buy you infosec. Not an infoturd.
Failing is easy
It is easy for infosec experts – or even experienced sysadmins – to look at that headline figure and ask incredulous, mocking questions. Many of us feel we could design a significantly more capable system for a fraction the cost, and the truth is that many of us probably could. Unfortunately, this doesn't take into account the realities of large scale infosec projects, which have undermined nearly every known attempt thus far.
Cutting through the excuses, most infosec projects fail for one reason and one reason only: the person (or people) in charge think they know what they're doing when in fact they have no idea. They can't know. It isn't possible to know. The threat landscape of information security evolves so rapidly that no one person could ever know enough to design a digital defence net all on their lonesome.
Most sysadmins could build something better than the DHS, but that isn't a high bar. Designing an infosec solution to defend an enterprise is the task of a team of experts. Defending a nation requires the resources of entire industries.
To succeed at infosec you don't need or want to hire someone who knows "all there is to know" about infosec. Down that road lies failure. Instead, what you want is a human search engine. Someone who has a pretty good idea of the fundamentals but whose primary value is that they know what they don't know, but know someone who is an expert in solving a given problem.
Meld some minds
The goal isn't to reinvent the wheel, but to use the best COTS solutions available to meet some needs and cover the rest with organisation-specific automation, orchestration and middleware. Email filtering is a great example. At this point, why would anyone bother reinventing this? If you don't feel that a given solution is accurate enough, string a couple of them together. You'll get the coverage you need without wasting valuable resources. Experts exist for this and they have commercial products. Let them do their job.
Infosec is also inextricably linked to network architecture and design. It needs to take into account everything from intrusion detection to surviving natural disasters. Edge defences to behavioural analysis and backups to test and dev. Infosec is as much about operational considerations as gizmos and gear, and requires a holistic approach to the totality of IT.
If that all sounds a little fluffy and theoretical, there are hard realities worth putting into practice. The first is never source all your infosec gear from one vendor. A single vendor is a single vision. A single vision for infosec means blind spots will occur and therein lie your vulnerabilities.
Modern infosec needs to have solutions from multiple vendors working in concert and orchestrated by experts. At least some of those experts need to come from outside your organization. Diversity of experience matters.
Above all, infosec should never become a bureaucracy. Bureaucracy is pork barrel politics, which is how infosec becomes a boondoggle. Infosec is a cage match: you adapt – and adapt quickly – or you die. Anyone who can break something has earned a seat at the table. Everything is worth consideration and everyone's ideas are valid until proven otherwise.
If you can't design and implement your own infosec solutions, don't be ashamed. Very few organizations have the broad expertise required to do so. Reach out to your vendors for help, but choose wisely. Choose a vendor that understands IT exists in your datacenter as well that of service providers and public cloud vendors.
Choose a vendor that understands solutions need to be fit to the customer; the customer doesn't need to be fit to the solution. Above all, choose a vendor that can and does look beyond "not invented here" syndrome and embraces the expertise of others to build the solutions required.