Learn Terraform with GCP — Part 1
Hashicorp has provided a very detailed set of tutorials on their website for anyone pursuing the Terraform Associate certification.
However, all of the code and directions use AWS environment and resources as examples. Although working with a specific Cloud provider has little to do with learning Terraform, my obvious bias towards Google Cloud Platform (GCP) led me to putting together a sample code base and instructions to learn Terraform with GCP resources used as examples.
So we start with ‘Build Infrastructure’.
Hopefully, you have a version of Terraform downloaded from Hashicorp in your computer.
If you haven’t, don’t worry — assuming you have access to a GCP environment, you don’t need to install Terraform locally on your machine. Click the ‘Cloud Shell’ icon on the top banner of the console to open your own linux virtual machine which is free for use up to 60 hours in a week.
5GB of storage is assigned to your home directory and the contents will persist even if you detach and reconnect to the Cloud shell.
Caution: Any other files or changes elsewhere in the filesystem will be lost on reconnection, so stay in your home directory and subdirectories when you create your code.
Cloud shell can automatically inherit your credentials from the GCP console, which will allow you to run Terraform with your own user permissions without any additional authentication. And did I say Terraform is already pre-installed in the Cloud Shell ! And it’s updated rather frequently so you have almost the latest version at any given point of time. A quick check on my Cloud shell showed version 1.2.4 is installed.
To follow this tutorial, you will need:
- Terraform CLI 1.2.0 +
- Google Cloud SDK installed locally or access to GCP Cloud Shell
- Proper IAM permissions in a GCP environment/project that allows you to create resources.
When possible, the configurations mentioned here use the free tier resources. But it is expected that you would be diligent in cleaning up your resources after training so as to avoid running up any costs in your environment.
Get Started
The set of files that are used to describe Infrastructure in Terraform is known as the Terraform configuration. This example guides you through creating a single GCP Compute instance (virtual machine).
Start by creating your working directory and change to the new directory. Create a new empty file named main.tf
mkdir -p learn-terraform/gcp-instance
cd learn-terraform/gcp-instance
touch main.tfIf you are using the Google Cloud shell, you have nano or vim, the common editors available from the command line to edit your code. What’s even better is that the Cloud shell provides a code editor similar to Visual Studio code which is entirely browser based. Clicking on the ‘Open Editor’ button will open up the editor in the same browser tab that your Cloud shell is on (see image).
You can toggle between the Editor and Terminal using the ‘Open Editor’ button which will change to ‘Open Terminal’ when the Editor is active.
Open the new file main.tf in the Cloud shell editor (see image) and paste the code below. Enter the appropriate project, region, zone and network values for your environment and save the file.
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "~> 3.90.1"
}
}
required_version = ">= 1.2.0"
}provider "google" {
project = "your-project-id-here"
region = "us-central1"
} resource "google_compute_instance" "web_server" {
name = "web-vm"
machine_type = "e2-micro"
zone = "us-central1-a" boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
access_config {
# Optional
# Include this section to give the VM an external IP address
}
}
}
This configuration will deploy a GCP compute instance. It assumes that you have a project created in GCP and a default VPC network (that is usually created in every project unless explicitly disabled by an organization policy).
Terraform Block
The terraform {} block is used to define required providers and other settings that Terraform will use to provision your infrastructure. The source attribute defines an optional hostname, a namespace, and the provider type. In this example, the google provider's source is defined as hashicorp/google, which is shorthand for registry.terraform.io/hashicorp/google.
This block also allows you to set specific version constraints for each provider. The optional version attribute, allows you to pin the provider version that has been verified to work with your configuration. If you do not specify this attribute, Terraform downloads the latest provider version available at that time.
To learn more, reference the provider source documentation.
Providers
You can define multiple provider blocks for all the resources required in your configuration. A provider is a plugin that interacts and operates with the corresponding Cloud, Infrastructure or application platforms supported by Terraform.
Resources from multiple providers can be used together within your Terraform module.
Resources
The resource blocks are used to define the individual components of your Infrastructure. In GCP, this may be a Compute Instance, a Virtual Private Network (VPC), Storage bucket, Cloud Load balancer or any supported resource available in the Google provider.
Resource block definitions have a resource type and the resource name. In this example, the resource type is google_compute_instance and the name is web_server. Typically the resource types are prefixed with the provider name. In the example, Terraform manages the google_compute_instance resource with the google provider. The unique ID for a resource is formed by the resource type, followed by a dot (.) and the resource name — for example, the ID for your Compute instance is google_compute_instance.web_server.
Resource blocks contain all the arguments needed to define a resource. This may include things like machine type, disk images or attached subnet. The providers reference list at Hashicorp lists the required and optional arguments for each resource. In this example configuration, the boot disk image is set to debian-cloud/debian-9, and the instance type is set to e2-micro which may qualify for the GCP free tier. Other parameters like the instance name, zone and network information is also defined in the code.
Initialize the directory
Deploying Infrastructure with Terraform begins with initializing terraform in your source code directory with terraform init. This command reads the code in the current directory, downloads and installs the required providers specified. In this example, the google provider is downloaded. In case you have not specified a provider type in your code, Terraform can deduce the required providers based on the defined resources. However it is best practice to define all required providers in the terraform block.
terraform initInitializing the backend...Initializing provider plugins...
- Finding hashicorp/google versions matching "~> 3.90.1"...
- Installing hashicorp/google v3.90.1...
- Installed hashicorp/google v3.90.1 (signed by HashiCorp)Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Terraform downloads the google provider and installs it in a hidden subdirectory of your current working directory, named .terraform. The terraform init command prints out which version of the provider was installed. Terraform also creates a lock file named .terraform.lock.hcl which specifies the exact provider versions used, so that you can control when you want to update the providers used for your project.
Format and validate your configuration
The terraform fmt command automatically updates the configuration in the current directory for readability and consistency. The command will print out the names of the files that it updated.
terraform fmtmain.tf
Running the terraform validate command will check to make sure your configuration is syntactically valid and internally consistent. If there are no errors, Terraform will return a success message.
terraform validateSuccess! The configuration is valid.
Create Infrastructure
Apply the configuration (create/update/delete the specified resources) using the terraform apply command. The output is similar to what is shown below. Some of the content was removed for brevity.
terraform applyTerraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ createTerraform will perform the following actions:# google_compute_instance.web_server will be created
+ resource "google_compute_instance" "web_server" {
+ can_ip_forward = false
+ cpu_platform = (known after apply)
+ current_status = (known after apply)
+ deletion_protection = false
+ guest_accelerator = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ label_fingerprint = (known after apply)
+ machine_type = "e2-micro"
+ metadata_fingerprint = (known after apply)
+ min_cpu_platform = (known after apply)
+ name = "web-vm"
+ project = (known after apply)
+ self_link = (known after apply)
+ tags_fingerprint = (known after apply)
+ zone = "us-central1-a"<content removed for brevity>
Plan: 1 to add, 0 to change, 0 to destroy.Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.Enter a value:
Terraform prints out the execution plan which describes the actions that it will take in order to change your infrastructure to match the configuration.
The output format is similar to the diff format generated by tools such as Git. The output has a + next to google_compute_instance.web_server, which means that Terraform will create this resource. This is followed by the attributes that will be set.
Values displayed with (known after apply) means that the value will not be known until the resource is created. For example, GCP assigns an instance_id to instances upon creation, which Terraform does not have the value for until the changes are applied and the Google provider returns that value from the API.
Terraform will now pause and wait for your approval before proceeding. If anything in the plan seems incorrect or dangerous, it is safe to abort here before Terraform modifies your infrastructure.
If the plan is acceptable, type yes at the confirmation prompt to proceed. Executing the plan may take a few seconds as Terraform waits for the GCP Compute instance to become available.
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ createTerraform will perform the following actions:# google_compute_instance.web_server will be created
+ resource "google_compute_instance" "web_server" {
+ can_ip_forward = false
+ cpu_platform = (known after apply)
+ current_status = (known after apply)
+ deletion_protection = false
+ guest_accelerator = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ label_fingerprint = (known after apply)
+ machine_type = "e2-micro"
+ metadata_fingerprint = (known after apply)
+ min_cpu_platform = (known after apply)
+ name = "web-vm"
+ project = (known after apply)
+ self_link = (known after apply)
+ tags_fingerprint = (known after apply)
+ zone = "us-central1-a"<content removed for brevity>Plan: 1 to add, 0 to change, 0 to destroy.Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.Enter a value: yesgoogle_compute_instance.web_server: Creating...
google_compute_instance.web_server: Still creating... [10s elapsed]
google_compute_instance.web_server: Creation complete after 12s [id=projects/**********/zones/us-central1-a/instances/web-vm]Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
The output indicates that your resource (Compute Instance) was created. Navigate to Compute Engine section in your project in GCP console to view the created instance.
Inspect State
Terraform creates a file called terraform.tfstate in your directory to store the ID and properties of the resources that it created. This state file is referred whenever you make any updates to the code and attempt to apply the changes.
The state file may contain sensitive information that you have provided as variables in the code or generated as outputs from a resource creation. Hence it must be stored securely and access restricted to authorized users only. Storing the state files in locations secured by authentication is a way to share it with teams working on the same code base. Terraform Cloud and Enterprise versions provide secure encrypted state file storage — other remote state backends like GCP storage buckets and Amazon S3 also provide secure and encrypted state storage.
The terraform show command allows you to view the current state. See below for the Terraform state for the above example.
terraform show
# google_compute_instance.web_server:
resource "google_compute_instance" "web_server" {
can_ip_forward = false
cpu_platform = "AMD Rome"
current_status = "RUNNING"
deletion_protection = false
enable_display = false
guest_accelerator = []
id = "projects/***********/zones/us-central1-a/instances/web-vm"
instance_id = "1565378762075647918"
label_fingerprint = "42WmSpB8rSM="
machine_type = "e2-micro"
metadata_fingerprint = "H6vdRThR6P0="
name = "web-vm"<content removed for brevity>scheduling {
automatic_restart = true
min_node_cpus = 0
on_host_maintenance = "MIGRATE"
preemptible = false
}
}
terraform state command also allows you to list the contents of the state file .
terraform state list
google_compute_instance.web_serverThe terraform state command also allows you to perform operations such as moving or removing individual resources from the state or migrating state files between local/remote backends. It is not recommended to manipulate individual resources in the state file outside of the deployment cycle, but these commands can come in handy when you have some unexpected state drift due to resources being modified outside of Terraform.
Troubleshooting
Even after running terraform validate on your code, there could be run time issues specific to the provider or incorrect data values that can cause your terraform apply to fail.
Using the sample code above in GCP, a scenario that you may run into is not having a default VPC in your current project where you are deploying the compute instance to. This will be encountered by the provider plugin when attempting to connect the instance to the specified network and cause the deployment to fail.
If this happens, you may need to create a VPC named ‘default’ in your project in automatic mode. Instead, you may also choose to define an existing custom VPC and subnetwork name as shown below, if you already have a VPC and subnet provisioned manually.
network_interface {
network = "my-vpc"
subnetwork = "my-uscentral-subnet"
access_config {
# Optional
# Include this section to give the VM an external IP address
}
}Save these changes and run terraform apply to redeploy the instance to the appropriate network. You will need to do the same on any following tutorials to this, since the sample code will be including references to the default network.
In Part 2, we will continue on to modify the infrastructure that you created with Terraform.
