Contact us
Contact us

Terraform Modules Overview

Learn how modules can solve configuration problems, how Terraform modules are structured, and how to create and use modules effectively in this tutorial.

post

IT Outposts » Blog » Terraform Modules Overview

Terraform Modules Overview

Managing infrastructure-as-code with Terraform is great — until suddenly, it’s not. What starts as a simple set of configuration files can quickly turn into an intricate web of resources, variables, outputs, and providers.

To simplify your operations, break up your configuration into reusable, composable pieces called modules. With them, your Terraform code evolves in an understandable, organized way, even as complexity increases.

This article covers the module basics you need to level up your Terraform skills. You’ll also learn module best practices to apply them right away.

Is Terraform a Configuration Management Tool?

Terraform is sometimes confused with configuration management tools like Ansible, Puppet, or Chef. But Terraform actually solves a different problem — infrastructure provisioning and management.

Configuration management tools focus on controlling resource settings once infrastructure already exists. For example, Puppet is helpful for ensuring that packages, files, and services are set up correctly on hundreds of virtual machines (VM). It covers the “day 2” management of existing resources.

Terraform instead tackles the “day 1” challenge of actually creating reusable infrastructure in the first place — whether on-premises or in a cloud. It provides a consistent way to declare and deploy the networks, instances, and services needed for an application to AWS, Azure, etc.

We often see IT leaders bring in IT advisory services to help their teams ramp up best practices on Terraform. The consultants recommend complementing Terraform’s provisioning strengths with ongoing configuration management for comprehensive lifecycle automation.

So, why does Terraform get labeled as a configuration management competitor? Part of it is the “infrastructure as code” overlap. Like Puppet, Terraform enables infrastructure to be codified and shared instead of relying on fragile manual processes. Teams can check in the configs to version control and collaborate on infrastructure changes, just like application code.

But Terraform takes a different approach, suited to provisioning rather than configuring. Instead of pushing settings to resources imperatively, Terraform generates ideal infrastructure plans from declarative descriptions and then reconciles any drifts toward the desired end-goal state. 

For example, a Terraform config would define “I need a VM instance with these disks, interfaces, and tags.” It doesn’t focus on how to get there — Terraform handles installing and configuring everything behind the scenes. This makes infrastructure more portable and repeatable compared to instance-by-instance scripts and commands.

Now, tools like Terraform and Puppet complement each other nicely once understood. Terraform initially sets up the scaffolding — it “builds the house.” Puppet then customizes and operates the environment — it “arranges and manages the furniture.” One focuses on provisioning infrastructure, the other on ongoing configuration.

So, what does Terraform mean? Let’s give it a clear definition.

The Definition of Terraform

Terraform is a software tool created by HashiCorp that lets you describe the resources your applications need and then build and manage them automatically. Instead of clicking around cloud consoles or running CLI commands, you can define infrastructure in simple configuration files that others can read and edit. It works across all the major cloud platforms, like AWS, Azure, and Google Cloud. No wonder nearly 35,208 companies currently use Terraform. 

The key benefit is avoiding the headache of manually configuring your infrastructure components. Do you want developers to spend days scripting up VMs and networks by hand, server after server? That’s what Terraform handles for you — just describe the desired infrastructure and let Terraform do the provisioning automatically.

In this way, Terraform allows you to create increasingly complex configurations as you manage your infrastructure with it. Terraform configuration files and directories don’t have an intrinsic complexity limit, so you can continue to write and update your configuration files in the same directory. Nevertheless, if you continue, you may run into the following problems:

  • You’ll have a much harder time understanding the configuration files.
  • Configuration updates will become more complicated since one section’s update can have unintended consequences for other sections.
  • As you configure separate development, staging, and production environments, you will have more duplication of configuration blocks, which will cause an increasing amount of burden when updating your configuration.
  • Sharing configurations between projects and teams is extremely error-prone and difficult to maintain if you cut and paste blocks between projects.

This is where Terraform modules can assist by encapsulating reusable, composable blocks of infrastructure that can be shared, versioned, and maintained efficiently across projects. To understand exactly how they can assist you, let’s first define Terraform modules.

What Are the Terraform Modules?

 Terraform modules are reusable packages of Terraform configuration code that allow you to break down your infrastructure into smaller, manageable pieces. Instead of defining everything in a single directory, you can develop components like networking, computing, storage, etc. and combine them together.

Modules operate like building blocks for your infrastructure. For example, you can create a Terraform module that provisions an Amazon EKS cluster. This Terraform EKS module would contain everything needed — VPCs, subnets, auto-scaling groups, etc. There are also useful Terraform AWS modules, like the Terraform VPC module, for creating reusable VPCs and networking components.

It is possible to create a simple set of Terraform configuration files in the following format:

Terraform Modules Overview
A simple set of Terraform configuration files

In this case, the contents of the minimal-module directory are taken into account as the root module when running terraform commands from that directory.

Calling modules

There is only one directory in which Terraform commands will use the configuration files directly, which is typically the current working directory. Using module blocks in your configuration, you can call modules located in other directories. Terraform loads and processes the configuration files associated with a module block that it encounters.

When one configuration calls another, it is sometimes called a “child module” of that configuration. 

Remote and local modules

It is possible to load modules either from the local file system or from a remote source. Several remote sources are supported by Terraform, including the Terraform Registry, most version control systems, HTTP URLs, and private module repositories on Terraform Cloud and Terraform Enterprise.

What Are Modules Used For?

With Terraform modules, you can abstract some resource sets logically. Modules are a means of grouping resources together and using them again and again.

We’ll consider a cloud-hosted virtual server with a few features. How might one describe that server based on its resources? As an example:

  • A virtual machine (VM) created from an image
  • For additional storage, a block device with a specific size 
  • A static IP that is also public, and is assigned to the server’s virtual network interface
  • To attach to the server a set of firewall rules
  • Additional block devices, additional network interfaces, etc.
Terraform Modules Overview
The use of a module “server”

In this case, let’s assume you’ll be building the server with a set of resources regularly. Modules are an excellent resource for you here.

In this example, we’ll show how this “server” module can be called.

In a configuration file, calling a module means using it.

Here, we create five instances of “server” using a single configuration (in the Terraform module example):

Terraform Modules Overview
Module Server: code example

Why Do We Use Modules?

Terraform Modules Overview

Starting with its common files, such as main.yml, might be a good place to begin working with Terraform. Particularly when you initially have a simple infrastructure, it is easier to fall into the “we’ll fix it later” trap. The problem with working against the clock (infrastructure, operation, and reliability engineers) is that you will not be able to spend much time refactoring code in the future. In the process of writing your code, you should be considering the quality and best practices. 

This is important, so let’s explain why: As your infrastructure grows (or as it gets more complex), you will face more challenges. In this case, your infrastructure will need to be scaled. Engineers face the challenge of scalability on a daily basis. It is very difficult to maintain and utilize a codebase that is intricate without any need for it. 

If you can’t reuse your code, you’ll have a hard time with maintenance. 

So, you will need to spend a lot of time studying the code, modifying, testing, and deploying it. These things should happen automatically through a CI/CD pipeline, but you will still have to wait. All engineers and end-users/customers are forced to pay a fraction of this overhead.

How is the Module Organized?

Hire a team of DevOps engineers with IT Outposts

Contact Us

Creating multiple modules makes sense. Here are a few examples:

  • An example would be a virtual private cloud (VPC)
  • static content hosting (i.e. buckets)
  • Load balancers and their resources
  • Configuration logs
  • As well as any other logical component of the infrastructure that you consider crucial

Imagine there are two modules: one called “server” and another named “network”.The module named “network” is where we identify and configure our virtual network and place servers in it:

Terraform Modules Overview
Two Examples of Module Structures

The custom modules we create can be referred to as “child” modules. A configuration file that refers to child modules is part of the root module.

Terraform Modules Overview
A diagram of the connection of child modules with the root module

It is possible to source a child module from the following sources:

  • local paths
  • the official Terraform Registry – this is similar to the Docker Registry, so you know what it is
  • a Git repository (a custom one or GitHub/BitBucket)
  • an HTTP URL to a .zip archive with the module

Is it possible to pass details about resources between modules?

Servers need to be created as part of a network in our example. Is it possible to tell the “server” module to create VMs in a network that has been created within the “network” module?

Terraform Modules: Best Practices

Terraform Modules Overview
Best practices for using Terraform

With great power comes great responsibility. And Terraform modules let you harness the power of infrastructure-as-code in new, compelling ways. But they also open the door to some anti-patterns if used naively — technical debt, hidden dependencies, and snowflake configurations can gather like storm clouds around immature modules.

So, how do you stay ahead of the coming storm while enjoying all the native superpowers of modules? By adhering to established best practices from the module design community,

Follow these Terraform module best practices, and you’ll avoid issues as infrastructure grows in complexity:

  1. First, start with the end in mind when architecting modules. Split components along boundaries that map to team ownership and environments. For example, a networking module handles all subnets, route tables, etc rather than having a subnet module, route table module, and so on.
  2. Embrace composition over inheritance. Modules should encapsulate single-purpose blocks of infrastructure reusable in various compositions. This reduces coupling for more mix-and-match flexibility.
  3. Focus modules on abstraction. Modules abstract away implementation specifics through loose coupling, fixed inputs and outputs, and maintainability ownership boundaries. Don’t tightly couple parent and child modules through hard-coded references and values.
  4. Standardize modules through opinions and governance. Mandate approved foundational modules across the organization via testing and policy controls. This raises the overall quality bar by aligning teams with opinions and best practices baked into centralized modules.
  5. Include comprehensive inline comments when authoring modules, especially public ones. Comments indicate usage examples, input/output explanations, and general tips for consuming the module correctly. 
  6. Make modules configurable but also opinionated by default. Offer sensible defaults and options for everyday use cases while allowing power users to customize niche requirements through input variables and conditions.
  7. Back modules with automated testing suites to inspire trust in stability over time. Shared modules that lack tests become dangerous liabilities, prone to uncaught regressions as dependent configurations grow.

Approach modules as long-term, reusable extensions of your application architecture. The principles of encapsulation, abstraction, and composability will serve you well over time.

Conclusion

Terraform modules are a useful way to encapsulate and reuse infrastructure configuration code. By abstracting away repetitive details into modules, you can build more modular, maintainable, and testable Terraform code. Modules encourage separation of concerns, allow code reuse across different environments, and help create consistent and predictable infrastructure.

While modules take some effort to learn and create, they pay dividends in the long run by reducing duplication and enabling collaboration. As with any abstraction, they hide some complexity, which can make debugging tricky at times. Overall, though, modules are an essential part of scaling Terraform usage across large projects and teams.

We’re happy to answer any questions you may have. With a wealth of tools and technologies, IT Outposts can optimize your business’ success.

FAQ

There are several good places you can share Terraform modules for others to use. Two top options are the public Terraform Registry and provider-specific module registries. These allow people to search for reusable modules with tags and details. You can also publish modules to open-source code repositories like GitHub so others can reference them from Terraform configurations.

Downloading Terraform modules is easy once you know where they are hosted. If a module is hosted in the public Terraform Registry, you can reference it by its registry namespace and name directly in your Terraform code. Terraform will then automatically download and use that module.

For modules shared in other ways, like on GitHub, you need to provide the full GitHub repository link. Terraform will clone the Git repo as needed to access the module.

Referencing a Terraform module means calling it from inside a root Terraform configuration. You can do this using a module block, specifying the source of the module and any other settings. For example, to use my-module located in a local folder called my-modules, you would add a module “my-module” block with the source set to “./my-modules/my-module.” Simple!

Click to rate this post!
[Total: 3 Average: 5]