The accelerating move to cloud computing over the past few years represents a major shift in the IT world. Driven in part by the lower total cost of ownership and flexibility, an increasing number of small and large organisations and companies globally are now utilising cloud-based platforms to host their infrastructure and utilising tools such as Terraform to deploy and manage it. This blog explores the benefits that tools like Terraform bring over cloud-native infrastructure management tools.

The benefits of moving to cloud computing are numerous, and they are tangible. For example, instead of waiting for months, it is now possible to provision a server and deploy production code in a couple of days. This not only saves companies from huge upfront costs but also provides a degree of flexibility not seen before when it comes to scaling IT infrastructure on-demand.

Rather than owning the infrastructure, companies are renting access to anything from applications to storage to compute resources from a cloud service provider.

Notwithstanding these benefits, cloud computing also brings with it a number of challenges. If these challenges are not addressed, the main benefits dissipate in the long run.

One of the main challenges, amongst others, is to deploy and manage cloud infrastructure proactively and effectively. To achieve this, a strong strategy, planning, and the right tooling are needed.

With the rise in the use of cloud computing, IaC (Infrastructure as code) has also gained a lot of attention and traction. This is a well-articulated definition of IaC available on the internet:

“IaC is a process of managing and provisioning mechanisms for authenticating, planning and implementing servers and data centers in the cloud and private network through machine-readable and understandable configuration files rather than physically configuring the hardware”.

Most cloud providers have their own set of tools and services for IaC. In this crowded market, a common approach was needed that could be used across the board.

Here comes “Terraform”, a tool that was first released in 2014 by a company called HashiCorp. With a large array of benefits and flexibility, Terraform is currently the most preferred tool globally for managing infrastructure in the cloud. 

What benefits does Terraform bring over cloud-native infrastructure management tools?

1. It is cloud-agnostic

A major benefit of Terraform is that it is cloud-agnostic, which means that it works with almost all cloud providers, including AWS, Azure, GCP, Alibaba etc. Once engineers are comfortable with Terraform, they can manage infrastructure on any cloud platform. This provides flexibility for the adoption of multi-cloud architecture, which is now desired and acceptable by most companies to obtain the biggest benefits from cloud computing.
The image below from HashiCorp demonstrates how flexible terraform can be with multi-cloud and other providers. More details about providers and supported CDK (Cloud Development Kit) languages can be found here and here.

 

2. A rich feature set provides flexibility

Terraform is rich in features, including Blocks, Comments, Arguments, and easy-to-use built-in functions. The use of variables, locals, and data sources provides flexibility to write clean, small, and reusable code. The major benefit Terraform comes with is its use of Modules and Templates. A Terraform module allows the grouping of resources and reusing them to make code less complex and repetitive. In many ways, Terraform modules are similar to the concepts of libraries, packages, or modules found in most programming languages, and provide many of the same benefits. These modules can be stored either in the local filesystem or can be shared using public or remote sources like Terraform Registry, Version control systems, HTTP URLs, etc. This saves engineers from writing repetitive codes and provides the opportunity to share their reusable codes among different teams, which in the end saves time and money.

3. Easy formatting and validation of code

As I said earlier, Terraform is rich in features. All these features make it easier for developers to write and manage code. Its built-in command “terraform fmt” allows the formatting of code to automatically update configurations in the current directory for easy readability and consistency. Similarly, to make sure a configuration is syntactically valid and internally consistent, “terraform validate” command is used. If there is any syntactical issue with a configuration then this command can help locate errors at an early stage.

4. Validate policies with OPA (Open Policy Agent)

Imagine how beneficial it would be if along with syntactical validation it is possible to validate business policies before provisioning cloud resources.

For example - A company wants all its compute resources to have specific labels or tags when created in different environments and, based on those environments, specifically compute resource types should be allowed. If the infrastructure request does not satisfy these conditions, then Terraform should not create the requested resources.

It is possible to achieve the above in Terraform using OPA (Open Policy Agent). OPA is an open-source policy engine, that allows the user to set policy as code, using the declarative language (Rego), and enforcing it across the entire stack. These policies can be validated in CI/CD pipelines against the JSON files which “terraform plan” creates.

Essentially, it is possible to enforce business policies (which security and compliance teams appreciate) by writing them in plain English (almost) and automatically with the help of CI/CD pipeline tools or IaC/CM tools without sacrificing availability or performance.
The image below shows how OPA works with JSON data to validate the business policies.



A good example of OPA with Terraform to apply policies can be found here: OPA Terraform Example.

5. Unit Testing

Testing is a major aspect when writing any piece of code to check if the code works as expected in all scenarios. This is particularly true in the case of infrastructure, where a small error could destroy an entire environment in a matter of seconds. It is highly recommended to test IaC and make it mandatory to include test cases for IaC.

“You can think of Terratest as a way to do end-to-end, acceptance or integration testing, whereas most other tools are focused on the unit or functional testing”. [1]

Writing test cases using Terratest is not hard and complicated. The procedure for writing unit tests follows the same setup and conventions of writing any Go unit tests which makes it easy for developers to learn and write their test cases.

6. Easy resource state management

Any IaC tool must store the state of managed infrastructure and configuration to keep track of resources created by the configuration and map them to real-world resources.
By default Terraform stores information about infrastructure state in a local file named "terraform.tfstate”. This state is used by Terraform to map real-world resources to the configuration and to keep track of metadata of resources so that Terraform can identify any changes in resources at a given point in time.

With Terraform it is easy to manage the state of the resources locally or remotely. To manage the infrastructure more efficiently storing the Terraform state remotely is the preferred approach so that multiple teams can work together without hampering the environment as all will be referring to the same state file. Current changes to the resources will therefore always be visible to everyone.

7. Easily preview and validate infrastructure changes before they are applied

Infrastructure topology is very complex without measuring and understanding the impact of potential changes. For a solution, it is essential for the DevOps team to know exactly what it is in for ahead of time when configuration changes are applied.

8. The Terraform Command Line Interface (CLI) thankfully fits the bill.

It helps developers validate and preview any and all changes to the infrastructure before those changes are applied. Having this capability not only eliminates a significant risk and back-and-forth but also allows developers to validate changes more effectively as they have full transparency and an easier understanding of how the proposed changes will impact the environment. Developers can also catch unintended changes earlier, rather than too late.

Further, there are some commands and tools available to create a visual representation of the configuration or the set of planned changes. The “terraform graph” command is used to generate the config graphs, and a tool like GraphViz can then be used to convert these graphs into a more human-readable format. [1]

9. Easy CI/CD

Terraform code can be easily stored in any VCS like GitHub and can easily build and deployed using any CI/CD tool like Jenkins or any Cloud-native CI/CD tool like AWS CodeBuild & CodeDeploy, Azure DevOps, etc. This gives the flexibility to use existing CI/CD tools to deploy for terraform scripts and automate the whole IaC process.

Wrapping Up

Terraform is a powerful and flexible tool, as outlined above, and works well with multiple clouds compare to cloud-native IaC tools. Its broad capabilities and flexibility continue to amaze and expand its user base. Terraform is open source and community support for it is strong and very active. It has been a helpful tool in my personal cloud journey with multiple cloud providers.

References

• https://www.terraform.io/intro/index.html
• https://www.terraform.io/docs/cli/index.html
• https://www.openpolicyagent.org/docs/latest/terraform/
• https://terratest.gruntwork.io/