Automatic join of Windows machines to an AD domain in Google Cloud Platform

With the growing prevalence of cloud in general, specifically the Google Cloud Platform, more and more enterprises are expanding or migrating their Windows environments to Google Cloud. There are a few Windows specific processes that administrators have to manage on a daily basis — one in particular is making sure that a new Windows instance in the Cloud is joined to their Active Directory domain.

As enterprises moved from physical machines to virtual, the domain join process which was mostly manual in the physical world was eventually replaced by VMware or Hyper-V services that automatically rename a Windows instance and join it to AD. They did this using stored admin credentials and some combination of virtual machine agents and/or startup scripts.

This article will describe a similar method for joining your new instances to an Active Directory domain hosted or extended to a Google Cloud environment. This is loosely based off the Google Cloud documentation here, with some of my experiences in getting this to work in my lab environment.

I will use the below diagram to explain the various components of the design and where they would reside in my ‘sample’ Google Cloud organization.

Google Cloud components to join and instance to AD domain

The main components of this design are:

• A Shared VPC host project and multiple service projects that contain the domain controllers, virtual machines etc. Most enterprises want to separate the management of networking and firewalls from application developers and other enterprise users. Shared VPC is a commonly adapted model to maintain this level of administrative control on the network and firewall which making the common network infrastructure available for multiple teams transparently. (More on Shared VPC in another post)
• Cloud Function and Serverless VPC connector deployed in the Shared VPC Host project.
• Compute instances in Shared VPC Service projects with a sysprep specialization script that references the Cloud Function. The Cloud Function will return a Powershell script for the instance at startup, which is then utilized to perform the domain join function.
• Ideally, Cloud DNS forwarding will be configured to the domain controllers that would let any compute instance on the VPC to resolve the domain controller names and other internal DNS entries. This would also reside on the Shared VPC Host project.

Disclaimer: The Google documentation link above states that the domain controllers have to be in the Host project in a Shared VPC configuration, but I was able to configure my domain controller in a Shared VPC service project in my working configuration. The documentation may be outdated or support for Shared VPC has been added to Serverless VPC connectors.

So let’s dive into the details. As we go along, we will need the following environment variables. These variables are required for the gcloud commands that execute the various steps in this post. I have renamed some of the original variable names from the documentation so it makes more sense in the context of the process.

# Required for most scripts mentioned in this document
HOST_PROJECT_ID     : Shared VPC Host project
VPC_NAME            : VPC Name
# Required variables for creating Serverless connector
SERVERLESS_REGION   : Serverless Access connectivity region
SERVERLESS_IP_RANGE : Serverless IP subnet (Must be a /28)
# Required variables for Cloud DNS forwarding configuration
DNSZONE_NAME        : Cloud DNS Zone Display name
DNSZONE_FQDN        : Cloud DNS forwarding Domain FQDN
# Required variables for deploying Cloud Function
DOMAIN_PROJECT_ID   : Same as HOST_PROJECT_ID; The cloud function
                      Makefile refers the project with this 
                      variable.
AD_DOMAIN           : FQDN of AD domain 
AD_NETBIOS_DOMAIN   : NETBIOS Name of domain
AD_PASSWORD         : Plain text password for the CompRegister user
PROJECTS_DN         : LDAP DN for the top level OU that contains 
                    authorized Project OUs
# Required variables for creating a VM instance
VM_PROJECT_ID       : Project where VM instances are created
VM_SUBNET           : Subnet for attaching the VM instances
VM_REGION           : Region for VM instances
VM_ZONE             : Zone for VM instances

Prepare Active Directory

• Login as a user with proper permissions to manage AD (typically Domain Admin) to a domain controller or any machine joined to AD and contain the Active Directory Administration tools.
• Create an Organizational Unit (example: Projects) in the domain. Assign the distinguished name for this OU to the PROJECTS_DN environment variable. Also set this value to a Powershell variable named BaseOU for assigning permissions.
• Create an AD account (CompRegistrar), assign a password and note the password down for use in the Cloud Function. This will be assigned to an environment variable named AD_PASSWORD in the cloud shell where the Cloud Function is built from.
• Assign permissions to the above AD account using the Powershell script below. This assumes the BaseOU variable is set to the DN of the Projects OU.

& dsacls.exe $BaseOU /G $CompRegistrar”:CCDC;Computer” /I:T | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:LC;;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:RC;;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:WD;;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:WP;;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:RP;;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:CA;Reset Password;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:CA;Change Password;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:WS;Validated write to service principal name;Computer” /I:S | Out-Null
& dsacls.exe $BaseOU /G $CompRegistrar”:WS;Validated write to DNS host name;Computer” /I:S | Out-Null

• Create Organizational Units under the Base OU, with the names of projects that will host the instances authorized to join the specified domain. Any number of projects may be defined under the BaseOU by creating OUs with their names.

Prepare Shared VPC Host Project

• Enable the required APIs on the project if not already enabled.

gcloud services enable cloudfunctions.googleapis.com \
    cloudbuild.googleapis.com vpcaccess.googleapis.com \
    --project=$HOST_PROJECT_ID

• Enable Serverless VPC Access connector. This will enable the Cloud Function to communicate with the Domain controllers with the specified Serverless IP range as the source addresses. This provides private access from the Cloud Function to the domain controllers on the ports specified in the next section, which in turn enables it to add the computer account and reset password for the VM instance that needs to be joined to AD.

gcloud compute networks vpc-access connectors \
    create serverless-connector \
    --network=$VPC_NAME \
    --region=$SERVERLESS_REGION \
    --range=$SERVERLESS_IP_RANGE \
    --project=$HOST_PROJECT_ID

• Assign the Project Viewer, Compute Network User and Storage Object Creator roles for the Cloud Functions service account:
  (service-<project-number>@gcf-admin-robot.iam.gserviceaccount.com)

export PROJECT_NUM=$(gcloud projects describe $HOST_PROJECT_ID \ 
  --format="value(projectNumber)")
gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member=serviceAccount:service-$PROJECT_NUM@gcf-admin-robot.iam.gserviceaccount.com \
  --role=roles/viewer \
  --project=$HOST_PROJECT_ID
gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member=serviceAccount:service-$PROJECT_NUM@gcf-admin-robot.iam.gserviceaccount.com \
  --role=roles/compute.networkUser \
  --project=$HOST_PROJECT_ID
gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member=serviceAccount:service-$PROJECT_NUM@gcf-admin-robot.iam.gserviceaccount.com \
  --role=roles/storage.objectCreator \
  --project=$HOST_PROJECT_ID

• Enable Cloud DNS forwarding for the AD domain by creating a forwarding zone pointing to the Domain controller addresses for DNS resolution. Enter the IP address of your domain controller(s) in the command to replace the quoted section.

gcloud dns managed-zones create $DNSZONE_NAME \
    --dns-name=$DNSZONE_FQDN \
    --networks=$VPC_NAME \
    --forwarding-targets="<DC IP addresses separated by comma>" \
    --visibility=private

• Configure a firewall rule in the VPC to allow access from the Serverless VPC connector IP range to the domain controllers. The assigned ports are for LDAP (TCP/389), Kerberos (TCP/UDP 88) and Kerberos password change (TCP/UDP 464). This rule is based on network tags. A specific tag (example: domain-controllers) assigned to the domain controllers will allow you to allow the traffic to those instances only. Example below — replace the section in quotes with the appropriate network tag for your domain controllers.

gcloud compute firewall-rules create allow-krb-serverless-to-dc \
    --direction=INGRESS \
    --action=allow \
    --rules=udp:88,tcp:88,tcp:389,udp:464,tcp:464 \
    --source-ranges=$SERVERLESS_IP_RANGE \
    --target-tags="<tag here>" \
    --network=$VPC_NAME \
    --project=$HOST_PROJECT_ID \
    --priority 10000

Configure Cloud KMS in the Host Project

• Enable Cloud KMS API in the Shared VPC Host project:

gcloud services enable cloudkms.googleapis.com \
    --project=$HOST_PROJECT_ID

• Create a global Cloud KMS key ring and key:

gcloud kms keyrings create computer-registrar-keyring \
    --location global \
    --project=$HOST_PROJECT_ID
gcloud kms keys create computer-registrar-key \
 --location=global \
 --purpose=encryption \
 --keyring=computer-registrar-keyring \
 --project=$HOST_PROJECT_ID

• Create a service account that the Cloud Function uses:

gcloud iam service-accounts create computer-registrar \
    --display-name="Registrar for Active Directory" \
    --project=$HOST_PROJECT_ID
export REGISTRAR=computer-registrar@$HOST_PROJECT_ID.iam.gserviceaccount.com

• Grant the service account permission to use the Cloud KMS key for decryption:

gcloud kms keys add-iam-policy-binding computer-registrar-key \
    --location=global \
    --keyring=computer-registrar-keyring \
    --member=serviceAccount:$REGISTRAR \
    --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
    --project=$HOST_PROJECT_ID

• For each project that is authorized to allow instances to join the domain, add the IAM Compute Viewer role for the CompRegistrar service account. Example below. Repeat this command for all projects that require this permission assigned.

gcloud projects add-iam-policy-binding <project-id> \
    --member "serviceAccount:$REGISTRAR" \
    --role "roles/compute.viewer"

Deploy the Registrar Cloud Function

• Clone the Github repository that contains the cloud function code from a Cloud Shell session authenticated as a user with privileges to create Cloud functions in the Shared VPC host project

git clone https://github.com/GoogleCloudPlatform/gce-automated-ad-join.git
cd gce-automated-ad-join/ad-joining

• Build and Deploy the cloud function by running ‘make’ from the above directory. When you are prompted, type Y to confirm that Cloud Functions allows unauthenticated invocations because the function implements a custom authentication and authorization mechanism. The deployment can take a couple of minutes to complete.
• Test the deployment by invoking the cloud function using curl. This should output the Powershell script for the Domain Join process.

curl https://$SERVERLESS_REGION-$HOST_PROJECT_ID.cloudfunctions.net/register-computer

Create and Join an Instance to the Domain

• An instance can be created and joined to the domain using the gcloud command as follows. The most important step in this command is the metadata value, that provides a sysprep-specialize script that retrieves the AD join code from the cloud function and executes it. NetBIOS only supports computer names of up to 15 characters. If the host name of a VM instance exceeds this maximum length, the sysprep-specialize script automatically changes the computer name to a name that is unique and has less than 15 characters.

gcloud compute instances create <instance-name> \
 --hostname=<custom-host-name-optional> \
 --image-family=<os-image-family> \
 --image-project=<os-image-project> \
 --machine-type=<machine-type> \
 --no-address \
 --zone=$ZONE \
 --subnet=projects/$HOST_PROJECT_ID/regions/$VM_REGION/subnetworks/$VM_SUBNET \
 --project=$VM_PROJECT_ID \
"--metadata=sysprep-specialize-script-ps1=iex((New-Object System.Net.WebClient).DownloadString('$REGISTER_URL'))"

• Monitoring instance startup logs from the cloud shell can be done by running the following command. Press Ctrl+C to cancel the monitoring and return to the cloud shell. The complete process of starting up the machine and joining it to the domain can take from 3 to 5 minutes and requires multiple reboots, done automatically by the GCP agent installed in the image.

gcloud compute instances tail-serial-port-output <instance-name> \
    --zone=$ZONE \
    --project=$VM_PROJECT_ID

• Verify that the VM has joined AD by logging in to the instance with an authorized domain user credential. The computer account in AD for the instance will also contain the following LDAP attributes when successfully joined using this process:

LDAP attribute                    Value
msDS-cloudExtensionAttribute1     Google Cloud project ID
msDS-cloudExtensionAttribute2     Compute Engine zone
msDS-cloudExtensionAttribute3     Compute Engine instance name. 

That’s what it takes to get automated AD join for GCP instances.

There are quite a few steps to implement this, but once the components are in place, all you need is to add the “— metadata=sysprep-specialize-script-ps1” to an instance in an authorized project to initiate the automatic join process. This makes it easy to deploy machines using any Infrastructure automation tools like Terraform.

Important things to watch for:

• Cloud DNS forwarding — if you can’t resolve the domain controller names from a new instance using the metadata server, this will not work.
• Permissions for the Cloud function service account, specifically the KMS key permissions for decrypting the CompRegistrar password. And Cloud Function logs in Cloud Monitoring (formerly Stackdriver) gives you a decent level of information to troubleshoot. The serial port output of the instance when booting up shows very generic error messages.
• Serverless VPC connector: There is no quick way to test if this is working other than writing your own Cloud Function to essentially do a DNS and port check to the domain controllers. You don’t see the subnet assigned to the connector in your VPC configuration, other than the routes table. So you can’t really spin up an instance in that subnet to test connectivity.

At some point, I would like to put together the whole process except for the Powershell section into a Terraform module. Stay tuned.