This article is more than 1 year old
Clone it? Sure. Beat it? Maybe. Why not build your own AWS?
Anything Bezos can do, you can do better, right?
You can't move without IT companies telling you about the "amazing" new technologies and features they've just launched, how you can't live without them, and what a shock it is that you've managed all these years before they were developed. And of course the bigger the company, the more new stuff they tend to pump out and the more critical it is that you sign up NOW.
Take Amazon Web Services, for instance. An evening's meander through their leafy forest of announcements will find all manner of new stuff that's come out over the last few months. And right next to it is a plethora of information about how their infrastructure works, how it's put together, and the techniques they've used.
Thing is, though, witchcraft and wizardry aren't part of their approach to operating AWS. In its basic form it's a collection of tin, with power applied to make it go, air-con to keep it working, and electronic string to let the bits talk to each other.
Are they really doing things we couldn't – and don't – do ourselves?
The basics of the architecture
Let's trot through a few of the key points AWS references regarding its architecture. AWS talks of its Shared Security Responsibility Model, in which it secures the underlying kit but it's up to you to secure the stuff you put in it. Nothing unusual there – the same applies in the enterprise where the IT team can stitch things up tightly but are still at the mercy of users sharing passwords and the like.
AWS provides log aggregation, which is a tremendously fashionable concept in the outside world (I wish I were a SIEM system salesman right now), but again you can do this yourself – in the last couple of months I've seen implementations of both £100,000 security information and event management installations and open-source SIEMs of negligible price.
Data encryption's another staple item, but again there are on-premises SAN solutions for which enabling at-rest encryption involves merely ticking a box, hitting SUBMIT and watching the progress bar. And even relatively modern things like Web Application Firewalling (WAF) are self-doable (Sucuri is the one I've seen used most recently, for example).
Of course, you have the other basic stuff – a central management console, a virtualisation layer that presents multi-site resilient systems as a single virtual entity and storage options with different performance levels (and of course corresponding price levels) – in fact just a week before writing this AWS announced that their DaaS service was moving to SSD.
But think again of an in-house virtual installation: Hyper-V and ESXi both give you centralised management with multi-site virtualisation, and slap vCloud Director on top of your VMware world and you have full Virtual Data Centre capabilities. As for tiered storage: I've worked with several companies whose tiered storage has evolved through the process of old SAN kit being repurposed to a lower tier as new SAN kit is slipped in at the front end.
And even when you look at Amazon's AWS Marketplace, what you're seeing is basically a collection of stuff that has been bundled together to make it easy to deploy. Which you can do with templates in your home-grown world, either self-built or downloaded from the various web-based repositories. Not forgetting, of course, that so many apps are available as "virtual appliances" these days.
But the non-basic stuff isn't AWS-specific either. So, as part of a flurry of funky new stuff in November 2016, for example, Athena was launched – a SQL-based query mechanism for S3 storage volumes. Dig into it, of course, and you'll discover that it's actually based on Presto, an open-source big data tool.
On the same day that we learned of a number of new compute instances including the F1 type: on the face of it, it's super-advanced and niche, but when you check out the specs you discover it's Xilinx hardware that you could buy too, should you be so inclined.
It's not all going their way
Of course, the key point of using the likes of AWS instead of doing it yourself is that you get a load of bonus stuff into the bargain. Someone else manages the data centres, keeps the underlying infrastructure running, builds the images for you to download from the marketplace, scales up the power provision as demand grows. The capital cost of putting this all together is someone else's problem, and leaving it to them is a welcome relief to many.
But this doesn't make it a no-brainer to use theirs instead of having your own. First of all, as IT services become more and more commoditised, the cost of data centre space and IT hardware is becoming increasingly competitive, and the options for software procurement are more varied (the skew towards up-front licensing is reducing, with more and more subscription-based options for buying the software).
Importantly, though, the financial model of your organisation is also significant. Of three companies I've worked with recently one was very focused on cashflow: if they could rent something for a hundred quid a month that was preferable to buying it for a couple of thousand. The other two, however, both had models where capital purchases were way preferable to rental, which made the cloud a trickier proposition to get approved. So it's not necessarily an automatic choice to go for the cloud approach. Particularly if we hark back for a second to the fact that costs are dropping: the kind of specialist hardware underpinning an F1 instance sounds scary but is priced in the thousands, not tens of thousands, range.
Stuff that's easier internally
Next, there are some things you do in your on-premises setup that are actually pretty easy to do, and in the cloud will actually make them harder to support (even if only a little) and may add cost. Direct Connect is an example: although you can pay for direct connectivity into AWS, you probably already have direct connectivity to your own world and you don't charge yourself an access charge or a throughput cost for the switch port. Similarly, your directory service (which is most likely Active Directory): you already have it, and to stretch it into the cloud takes time and effort. And to go back to the WAF concept, it's almost as easy to whack in something on a local VM as to rent it from a cloud provider.
And stuff you can't avoid
Finally, there are some things that the cloud providers, AWS included, tout in their marketing blurb as essential but that you can't leave entirely to them – you still have to do it yourself. The big one here is compliance. Now, Amazon say – quite rightly – that they have "certifications from accreditation bodies across geographies and verticals, including ISO 27001, FedRAMP, DoD CSM, and PCI DSS". And you'd expect them to have these, of course.
What some people forget, though, is that just because you run your stuff in the cloud and the cloud provider has these accreditations, it doesn't absolve you from the responsibility of running your own ship well. Just as the Shared Security Responsibility Model has AWS doing a chunk of the work while you're responsible for the rest, the same applies to accreditations: doesn't matter that they have their ISO 27001 certificate if you're not implementing and following, say, an effective information security or access control policy. Yes, you're saving some effort by offloading the back-end tasks to them, but in some cases not a great deal.
Is it true, then, that anything AWS can do, I can do better? No. Yes, there may be a handful of things that are trickier, more expensive or a bit more involved than keeping things internal, but the benefits will generally be worth the effort and cost.
The point is, though, that even if you do go with the cloud, the chances are that you will still retain some on-premises infrastructure and that you won't shift your entire world into the cloud. So you can apply the same design techniques, security concepts and operational principles as the big cloud providers in order to gain similar levels of best practice and hence security and robustness.
And you can keep some of your work on-premises because sometimes it's just nice to be able to try stuff out without having to count the cost of spinning up cloud services and incurring per-hour charges – the certainty of a one-off purchase outweighs the uncertainty of a pay-as-you-go model. ®