If you're a developer, an IT infrastructure engineer, a manager working on some infrastructure automation project, a business professional looking for a lead to begin with the IaC journey, or even just a person keen to know about IaC, then you have landed at the right place!
This blog series will be about Infrastructure as Code, popularly known as "IaC". It will mainly revolve around the available tools, best practices, coding methods with examples, recent trends, and so on.
You can follow through this whole series or skip to the specific IaC topic which you are interested in or looking for.
This blog, being the second in the series, will be an introduction to the prominently available IaC tools. We will dive deep, and try to do a thorough comparison of these tools to help you pick the best IaC tool relevant to your project.
Table of Contents
- Which are some of the popular IaC tools and which one is for you?
There are a lot of IaC tools available in the market which we can use to provision and manage our infrastructure. But, choosing the right one for the project is a big task. It also depends on a number of factors, including:
- project requirements (Provisioning and management, or just provisioning)
- platform on which the deployment is supposed to be (AWS, Azure, On-premise, etc.),
- skillset available within the team (including language known for development),
- and so on...
Let us dig a bit deeper into the available IaC tools, what are the features available with them, and what makes them different from the others.
Which are some of the popular IaC tools and which one is for you?
Let us now look at the prevalent IaC tools and in what situation, which one would be the most productive tool to use. A thorough comparison will be helpful in selecting the IaC tool, which suits your project requirements.
Approaches to IaC
As we had seen in the last blog, there are 2 ways of writing the infrastructure as code:
Based on these approaches, the IaC tools work, and it allows us to decide which one to use for our project.
Terraform is the most widely used IaC tool as of date. It is a platform-agnostic open-source tool for infrastructure automation, offered by HashiCorp. It helps in configuring, provisioning, and managing the infrastructure with the code.
It can be used to do multi-cloud deployments as it supports almost all prominent cloud providers and even platforms like Kubernetes and Heroku. Nonetheless, it also allows the building of on-premise infrastructure.
Terraform configuration is written in a language, developed by HashiCorp itself, known as HCL (HashiCorp Configuration Language).
Terraform provides a CLI, and allows us to do a pre-check of the code by creating a plan, before deployment. This makes it easier to make sure if the configuration is as expected or not. The state can be saved as a file and stored on a disk, S3, source control, etc. Furthermore, it allows destroying the whole infrastructure in just one go.
HashiCorp also offers "Terraform cloud" which is a Saas-based version of Terraform, managed by themselves. It helps teams to use Terraform together. It manages and runs in a consistent and reliable environment, having disposable virtual machine instances that are deployed in their own cloud infrastructure.
- a set of language native frameworks for defining infrastructure
- adaptors to an underlying provisioning tool
The CDK for Terraform will generate the Terraform configuration using these libraries of AWS CDK. This change is quite good if you're a developer. Since, it allows using general-purpose programming languages like TypeScript, Python, Java, C#, and Go (experimental), to generate the Terraform configuration, and removes the need to learn HCL.
CloudFormation templates are written in either YAML or JSON format. It is limited to only the AWS platform, but since it has integration capabilities with other AWS resources, it makes it quite prevalent. It allows us to simplify AWS infrastructure deployment and management, replication across regions and accounts, and easily control and track changes.
In CloudFormation terminology, the related resources are combined together as a single unit, and is called a stack. Also, we can create "change sets", which is a summary of the proposed changes to running resources in a stack. CFn also uses rollback triggers to restore stacks to a previous state.
In the backend, CDK converts the general purpose programming language code into a CloudFormation template and deploys the resources on AWS.
So, the flow can be imagined like:
-> Write CDK Code in TypeScript, Python, etc -> CDK generates CloudFormation Template -> CDK deploys & manages AWS Resources
This is one of the best IaC tools for Developers, since they get the flexibility and expressive power of programming languages, and use software engineering practices to make the infrastructure more reliable and robust.
Azure Resource Manager
The Microsoft Azure platform provides an IaC tool called "Azure Resource Manager" to automate infrastructure deployment and management. It allows us to deploy, manage, and monitor the resources in the group.
Azure Resource Manager uses ARM Templates, which is defined in JSON format, to define and manage resources and their dependencies within the infrastructure. It is easy to use as well, which makes it quite prevalent in the industry.
Azure supports Role-based access control (RBAC) by default, which adds the feature of controlling access to the services and resources. We can use this to provide fine-grain access to the management groups, subscriptions, and resource groups. It also offers us to tag the resources to logically organize them in subscription, and check the costs of resources having a specific tag.
With Azure Resource Manager, it is quite convenient to quickly deploy the infrastructure multiple times throughout the development lifecycle, maintaining the resources in a consistent state.
Google Cloud Deployment Manager
Google offers an infrastructure deployment service known as "Google Cloud Deployment Manager" for its own Google Cloud platform. It is used for the automation of the creation, deployment, and management of Google Cloud resources.
Google Cloud Deployment Manager allows us to simultaneously deploy many resources. The configuration, in the form of code, is used as a single source of truth to maintain the infrastructure in a consistent state.
Puppet is one of the oldest configuration management tools, which has its own declarative language. It is yet another tool of infrastructure automation with a little infrastructure management capabilities.
Puppet has its own Domain-specific language (DSL), based on Ruby, called "Puppet Code", in which we can define the desired state of our infrastructure. The puppet ecosystem has a "Puppet Platform" consisting of "Puppet Primary Server" and "Puppet Agent", which are used to automate the infrastructure and maintain the desired state.
The Puppet primary server stores the code that defines the desired state. The Puppet agent translates the code into commands and then executes it on the systems we specify, in what is called a "Puppet run".
Puppet user interface is quite intuitive, as it allows us to monitor and manage the whole infrastructure in real-time through a single pane of glass. It can be used to automate infrastructure on almost all of the leading cloud IaC platform providers, including AWS, Azure, GCP, VMWare, etc.
Ansible is an orchestration and configuration management tool, which is used to automate the provisioning of infrastructure. Though, it focuses more on configuration management and infrastructure provisioning, not infrastructure management.
The Ansible code is written in YAML format, often called "Ansible Playbook", and is used to store the list of tasks for repeated execution on managed nodes. Ansible enables us to execute these playbooks to create the infrastructure with the required configuration. Ansible is agentless, and it performs the tasks by temporarily connecting via SSH or Windows Remote Management (WinRM).
Ansible is considered one of the simplest ways of infrastructure and application configuration management. Moreover, it allows us to write our own modules and plugins to extend the existing features as per the need. It supports both on-premise and cloud environments.
Pulumi is one of the latest Infrastructure as Code tools, and it has quickly captured the market with its approach of Developer-first and great flexibility. It is an open-source tool for creating, deploying and managing cloud infrastructure. It also supports containers, Kubernetes clusters, and serverless functions.
The pulumi CLI, runtime, libraries, and a hosted service work together to provision, update, and manage cloud infrastructure.
Comparison of IaC Tools
|IaC Tool||Terraform||AWS CloudFormation||AWS CDK||Azure Resource Manager||Google Cloud Deployment Manager||Puppet||Ansible||Pulumi|
|Type||Orchestration (IaC) Tool||Orchestration (IaC) Tool||Orchestration (IaC) Tool||Orchestration (IaC) Tool||Orchestration (IaC) Tool||Configuration Automation/Management Tool||Configuration Automation/Management Tool||Orchestration (IaC) Tool|
|Use Cases||Provision and manage major cloud and on-premise infrastructures||Provision and manage AWS infrastructure||Provision and manage AWS infrastructure using general-purpose programming language||Provision and manage Azure infrastructure||Provision and manage Google Cloud infrastructure||Provision and configure infrastructure on all major cloud and on-premise environments||Configure pre-existing systems and support network device automation||Provision and manage major cloud and on-premise infrastructures using general-purpose programming language|
|VM Provisioning, networking and storage management||Comprehensive||Comprehensive||Comprehensive||Comprehensive||Comprehensive||Partial||Partial||Comprehensive|
|Lifecycle/State Management||Lifecycle aware; Can save the state of deployment in S3, DynamoDB, locally, etc||Maintains state in AWS in the form of stacks automatically||State management through AWS CloudFormation under the hood automatically||Maintains state in Azure as a template automatically||Maintains state in Google Cloud automatically||Desired state is managed and maintained by the puppet primary server||It doesn't keep track of the state of the infrastructure||State can be saved locally, in S3, or Pulumi service backend|
|Command Line Interface||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Configuration Drift Detection||Yes||Yes||Yes||Yes||Yes||Yes||Yes, can be implemented||Yes|
|Change management||Using Terraform Plan||Using change sets||Using cdk diff||Using "what-if" operation||Using "--preview" flag||With agent-based drift alert||No||Using Pulumi Preview|
|Module support||Infrastructure can be reproduced with the help of modules||Allows references between stacks and nesting of stacks||Can create multiple stacks, nested stacks all in one project and refer internally as well||Multiple environments can be deployed using the template with different parameters||Templates can be reused across different deployments||Modules are available for reusability and shareability of the code||Reusable playbooks can be created||Infrastructure can be reproduced|
|Multi-cloud support||Yes||No, AWS only||No, AWS only||No, Azure only||No, GCP only||Yes||Yes||Yes|
|Pros||Works across muliple cloud providers; Integrates multiple cloud services with external functions; "Plan" phase feature to test proposed changes||Closely integrated with other AWS services; Automatic rollback of deployment failure; Management capability through the AWS Console||Flexibility of using any programming language; Able to detect drifts between code and environment||Closely integrated with other Azure services; Management possible through Azure Console||Closely integrated with other Google Cloud services; Allows to use Python to execute custom logic within template; Useful in managing Kubernetes state as well||Web UI console makes management easy; Very robust and has native shell constructs capabilities; Stable and mature system with active puppet community||Agentless operation; Capable in multivendor environment; Open source and vendor community support||Best in class Kubernetes support; Built-in secret management; Flexibility of using any programming language|
|Cons||No automatic rollback; Use of HCL can hinder adoption||Only available for AWS||Only available for AWS; Require programming knowledge; Feature lag sometimes||Only available on Azure||Only available on GCP; Not enough documentation||It's a pull-based system (agent) which increases time; Need CLI for advanced tasks; Puppet DSL can a bit complex to use||Slow when collecting large volume information; Functions are tailored for device-specific configuration||Not enough documentation|
|Pricing||Cloud; Enterprise; Open-source||Free||Free||Free||Free||Enterprise; Open-source||Standard; Premium; Open-source||Team; Enterprise; Business Critical; Open-source|
|Support||HashiCorp support and a large open-source community||AWS Support and a large open-source community||AWS Support and open-source community||Microsoft Support and open-source community||Google Cloud Support and open-source community||Puppet Support and open-source community||Red Hat customer support and open-source community||Pulumi Support and open-source community|
Infrastructure as Code has really become an integral part of almost all the cloud and on-premises infrastructure, and in turn, any software development lifecycle, in general.
And, it is really an important task to pick the right IaC tool for the project, considering all the requirements, limitations, and other important aspects of the project. These tools make the development, provisioning, and management of infrastructure easy and manageable for the teams working together.
We will try to cover some other IaC-related topics in the upcoming blogs. So, stay tuned!
Atul Anand, Fellow
JTP Co., Ltd.