In my last post I’ve complained about AWS native container management solution - ECS. Basically, it’s about lack of Service Discovery & Distributed Configuration as well limitations of Elastic Load Balancer and Cloud Watch (Logging). Doing the next logical step, in this article we look at alternative Container Managers. But let’s first recap why we need container orchestration…

Why container orchestration

As long you have to manage a handful of containers on one host you may ask this question as soon your application start to use several Databases, Loadbalancer, service workers, caches, or let’s say a lot of containers you quickly start to think about challenges like:

  • Service Discovery
  • Load Balancing
  • Auto-[scaling/restart/healing] of containers and nodes
  • Secrets
  • Configuration
  • Storage management
  • Health checks
  • zero-downtime deploys

Here is the list of Container Orchestration systems that provide solutions to all/some of these challenges.

Container Orchestrators list

Kubernetes

kubernetes - an open-source system for management of containerized applications. Build from scratch with learnings from Borg1. Kubernetes seems to have the most advanced integrated feature set (End of 2016). Scheduling, Service Discovery, Proxy & Load Balancing as well as a lot of other small things are built-in. And of course, Google Container Cloud (GCC) is Kubernetes driven.

Docker Swarm

Docker in Swarm Mode starts to step on Kubernetes feet from version 1.12.2. The build-in swarm mode allows the managing of clusters and services comparably to Kubernetes. Still, it doesn’t have a feature set of Kubernetes, but native to Docker and very comprehensive.

Nomad

HashiCorp’s do also offer a Docker orchestrator - Nomad I haven’t tried it yet.

Mesosphere

Mesosphere’s Datacenter Operating System (DCOS) advanced competitor of Kubernetes but it’s not a container orchestration tool per se, it is more than that, it is positioned like “Operation System for DataCenter. Where Marathon project is the Container managing part.

Docker Cloud

Docker Cloud (with AWS) can be seen as a logical extension of the existing Docker cloud product - docker hub. Similar to Empire but the Container is managed from the Docker Cloud, still Containers running on your (AWS) machines. I guess behind the scene Docker Cloud runs the latest docker in swarm mode and provisions your machines with Docker machine and InfraKit. In that case, Docker Cloud is a kind of nice UI wrap-up for console tools and a kind of logical integration to the Docker universe.

ECS + Custom Service Discovery + Custom Proxy

In my case this is would be a kind of evolutionary way to do it2. Still different topics that not solved (or not solved well) with ECS need to be configured and maintained by a set of other tools. Here is an excerpt of tools that are on my mind grouped roughly by purpose

Other options

  • Empire - open-source, self-hosted PaaS that makes deployment and management of dockerized 12-factor apps easy. According to the Documentation, the project tries to rebuild the Heroku experience.
  • Flynn Another PaaS for Containers.
  • DAIS similar to Flynn?
  • Shipyard Docker Swarm based Manager
  • Dokku not sure if it has a right feature set for being CaaS

And to make it complete here some more Lists:

Distributed Storage

  • Consul - brilliant distributed K/V storage, you can build a lot around this.
  • Etcd - small, distributed K/V storage. Used by Kubernetes
  • SkyDns - service for announcement and discovery via DNS (Build on top of etcd)

Registrators

Smart Loadbalancers

Please give me a hint to other good tools of this categories if you miss something

Summing-up

I’ve already had my favorites, which are Kubernetes and the newest Docker in Swarm Mode. They are absolutely worth getting to know them in detail and having hands-on them independently of what path my team chooses.

Highly appreciate any feedback on the current listing.

  1. Borg is the name of Google’s internal Container Cluster Manager. ↩︎

  2. Looks like this is the only advantage I see following this path. ↩︎