Recently I wrote about First steps with containers on AWS. Since then this project has grown a bit, but most of the resources were created manually. Early or later this needs to be automated.
Ansible
Since we had some experience with Ansible this was a first tool to try. I had first results relatively quickly. However, Ansible felt not the right tool for the job. The fact that you work with localhost destroys a lot of Ansibles charm that I saw while provisioning Virtual machines. Also Chef, Puppet also were created to provision things on hosts (not creating them) and are often classified as configuration management tools.
- hosts: localhost
connection: local
gather_facts: False
...As well the quality of the AWS modules was not that good, so that idempotent operations were too often not possible so in the end I felt not having enough control and decided to try other tools for AWS automation.
CloudFormation
So I turned to AWS native CloudFormation (CF). The documentation states, that it’s even possible to generate a CF description of the existing infrastructure… However while doing the first trials it turned out, that CF was not that reliable on the creation or on destroying of AWS resources. Something did not work to 100% with CF even when i played with kube-aws, that generates CF under the hood, and even with that CF AWS could not delete resources and could not provide proper feedback on the error.
Furthermore, CF is just a huge JSON (or YAML) that lists all the AWS resources you have or need. It’s hard to maintain at the end even if you can break it into several files as well. On one meetup people reflected the same experiences to me, so CF slipped down my priority list.
Terraform
Hashicorp’s terraform finds good responses on the internet, so I tried it and liked it for simplicity. The code seems to be even more declarative than Ansible. The concept of state makes it much more reliable and understandable. Code can be structured as you like and modules are supported. Modules a written in the same simple and well readable way. There are only few concepts to understand:
- providers
- resources
- data sources
- variables
- modules on top of all
Having that, you can read and understand on your own other people’s code from the first minute. Just by looking up documentation for parameter details. I also found it useful that it’s possible to “import” already existing AWS resources into the “managed state” even no resource definition is created it’s still helping.
//Example of VPC and IGW resource definitions
resource "aws_vpc" "mod" {
cidr_block = "${var.cidr}"
enable_dns_hostnames = "${var.enable_dns_hostnames}"
enable_dns_support = "${var.enable_dns_support}"
tags = "${merge(var.tags, map("Name", format("%s", var.name)))}"
}
resource "aws_internet_gateway" "mod" {
vpc_id = "${aws_vpc.mod.id}"
tags = "${merge(var.tags, map("Name", format("%s-igw", var.name)))}"
}So, terraformfeels like the tool of the choice for now. However, even terraform is not perfect. For example, I’m still not satisfied with environment separation concepts. Currently, I have to maintain every environment in my own folder.
Own tool for AWS API
Except for CloudFormaton, all other tools are using AWS API and do not suffer from any issues. Also, I have already used API for small monitoring tools and found it pretty comfortable. So it might be an option to create your own tool(s) that use AWS API in the programming language you comfortable with or that is more appropriate for your needs E.g. java or maybe better Go in case of a console tool.
Summarizing all that I think I can create a priority of the tools I would like to use for AWS automation:
- Terraform
- Ansible
- Own API console tool (Golang)
- CloudFormation
P.S. Existing infrastructure
An extra topic that adds much complexity to the topic is the existence of manually created resources. For example, I would like to include a lot of manually created AWS resources, that are used in production. Reasoning on this I see a fundamental problem, that manually created resources are tending to break patterns. And because of that, there is no matching community module for this and if you write your own, you probably will need to do a lot of compromises and comments in the code. And in the end, this infrastructure code tends to be ugly and strange.
By that maybe there is no sense at all in this kind of reverse engineering work of existing AWS infrastructure to code. At least not to all parts and it’s better to recreate the infrastructure from new code as far as possible and invest more time and care on smooth migration.
As always excited about your feedback.