EXPOSED: Amazon's cloudy Docker-friendly EC2 Container Service

We pry open the new AWS control freak to get to the guts

Analysis Amazon stoked the fires of Linux container fever on Thursday with the introduction of its new EC2 Container Service for the Amazon Web Services (AWS) cloud. But what's the real benefit of ECS and how does it work?

There's no need to ask who ECS's intended audience is. At the AWS re:Invent conference in Las Vegas this week, hundreds of attendees queued for as much as an hour to attend a session hosted by container maven Docker, and many were turned away at the door once the room filled. Containers, and by extension Docker, seem to have piqued the interest of nearly everyone in the Linux world.

But although Docker and other companies have made a lot of noise about how containers will revolutionize IT, migrating your existing infrastructure from traditional, monolithic server stacks to a container-based model is no easy task.

Keeping track of all of your containers, knowing where they're running and what they're doing, and scaling your fleet of containers to meet demand can all be challenging – which is where ECS comes in.

EC2 Container Service under the hood

At the highest level, ECS is a service that allows you to run and manage containers across a cluster of EC2 instances. So what's that, then?

First, for "containers," read "Docker images." In Amazon's view, containers and Docker are essentially synonymous, and each instance in an ECS cluster is running a Docker daemon to power its containers.

Amazon's Dan Gerdesmeier spun up WordPress in minutes at AWS re:Invent

Digging further in, AWS engineer Dan Gerdesmeier explained at re:Invent that the ECS model can broadly be broken down into containers, tasks, container instances, and clusters.

Containers, obviously enough, are the individual Docker images of the software you want to run. Tasks are simply groupings of related containers. For example, you might have one container that holds an Ngnix webserver, another that holds a MySQL database, and a third that holds a Ruby on Rails application. Individually they're just containers, but together they form a system that Amazon calls a task.

You set up the containers and tasks you want to run on ECS via JSON definition files. A container definition specifies an image and the AWS resources it needs, such as CPU, memory, and ports. A task definition is essentially a list of container definitions, plus a version number and a name for the whole family.

Moving further up the stack, you have container instances. Conceptually, these are virtual machine instances on which tasks are scheduled. Technically, they're EC2 instances that run Amazon's ECS Agent, which is a piece of software written in Google's Go language that provides the special sauce that makes ECS work.

"The EC2 Container Service Agent is responsible for checking the instance into your cluster," Gerdesmeier explained. "It performs all the heavy lifting of making sure that the containers on your individual instance stay up, and it reports all the information back to the service."

Finally, we have clusters, which are the uppermost level of the ECS stack – although they're also mostly conceptual. While ECS always spins up a cluster of container instances, it could be a cluster of one. Suffice it to say that a cluster provides the pool of AWS resources for your tasks to run on, however large or small.