Build your own Hosted VSTS Agent Cloud: Part 2 - Deploy

2018-02-27

In Part 1 you’ve seen how to use Packer to build a custom image based on a Packer configuration file with an Azure builder and create a new VM from the portal. In this blog post I’ll show you how to script a build and deployment based on your custom image and how to deploy the VSTS agent to it.

This post is the second in a series of 4:

Scripting your Packer build

On GitHub you can find the complete build script that I use. Here I just want to discuss how the script works.

First, I define all the input values I need to run packer build.


[CmdletBinding()]
Param(
   $Location = $env:Location,
   $PackerFile = $env:Packerfile,
   $ClientId = $env:ClientId,
   $ClientSecret = $env:ClientSecret,
   $TenantId = $env:TenantId,
   $SubscriptionId = $env:SubscriptionId,
   $ObjectId = $env:ObjectId,
   $ManagedImageResourceGroupName = $env:ManagedImageResourceGroupName,
   $ManagedImageName = $env:ManagedImageName,
   [switch]$InstallPrerequisites
)

In VSTS I created a Variable Group that contains all values I need to build and release my Hosted Agent. This makes it easy to share the variables between different build and release definitions. I can choose to use the variables from the environment variables created by VSTS or I can directly pass them to the script.

A Variable Group contains all values I need for building and releasing my Agents

I’ve added a switch to the script to install the prerequisites needed for building the Packer image. I first install chocolatey and then use that to install Packer and Git. I also install the AzureRM PowerShell commands. Make sure to run this with an agent that is an administrator otherwise the chocolatey install fails:


if ($InstallPrerequisites) {
   "Installing prerequisites"
   Set-ExecutionPolicy Bypass -Scope Process -Force
   Invoke-Expression ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

   "Install Packer"
   choco install packer -y

   "Install Git"
   choco install git -y

   "Install AzureRM PowerShell commands"
   Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force
   Install-Module AzureRM -AllowClobber -Force
   Import-Module AzureRM
}

The next step is removing the previous image to make room for the new one. I just check if the resource group where the image is stored exists and if it does I remove it. For this I use a handy PowerShell trick where I set both ErrorVariable and ErrorAction. ErrorAction SilentlyContinue makes sure that if Get-AzureRmResourceGroup throws an error because it can’t find the resource group, my script continues without showing an error. Instead it sets notPresent to the error text. I can then check if the notPresent variable is empty meaning that there was no error, the resource group exists, and I can remove it.


Get-AzureRmResourceGroup -Name $ManagedImageResourceGroupName -ErrorVariable notPresent -ErrorAction SilentlyContinue

if ( -Not $notPresent) {
   "Cleaning up previous image versions"
   Remove-AzureRmImage -ResourceGroupName $ManagedImageResourceGroupName -ImageName $ManagedImageName -Force
}

The following snippet is to make sure that after installing chocolatey I’m in the correct directory where I want to build my image. If $env:BUILD_REPOSITORY_LOCALPATH is set, I’m running a build through VSTS and I want to set my current working folder to it.


if ($env:BUILD_REPOSITORY_LOCALPATH) {
   Set-Location $env:BUILD_REPOSITORY_LOCALPATH
}

The VSTS Packer build creates a folder on the VM where it stores the commit id in a text file. This helps you easily check which version of the agent software a VM is running:


$commitId = $(git log --pretty=format:'%H' -n 1)
"CommitId: $commitId"

And then we’re finally ready to call packer build. I pass in all the arguments on the command line instead of using a configuration file. This allows me to store the configuration values in VSTS and makes the whole script more flexible.


packer build `
   -var "commit_id=$commitId" `
   -var "client_id=$ClientId" `
   -var "client_secret=$ClientSecret" `
   -var "tenant_id=$TenantId" `
   -var "subscription_id=$SubscriptionId" `
   -var "object_id=$ObjectId" `
   -var "location=$Location" `
   -var "managed_image_resource_group_name=$ManagedImageResourceGroupName" `
   -var "managed_image_name=$ManagedImageName" `
   -on-error=abort `
   $PackerFile

And that’s it. After running this script, you have your image.

Deploying your custom image

I’ve created a Release PowerShell script that you can find on GitHub. The script consists of two parts: VM deployment and VSTS Agent installation.

I’ve chosen to deploy my Virtual Machines to a Virtual Machine Scale Set. A VMSS helps with automatically scaling the number of Agents I want. By default, the script only deploys one Agent but with a simple command you can scale the agents up and down.

To create a VMSS you need some networking resources in place. Create a Virtual Machine Scale Set with Azure PowerShell shows all the commands you need to create your network and initialize your VMSS. These are all standard Azure actions, so I won’t discuss those here. One important step that I do want to highlight is getting a reference to the image that you’ve build and setting it as the image for your VMSS:


$vmssConfig = New-AzureRmVmssConfig `
    -Location $Location `
    -SkuCapacity 1 `
    -SkuName "Standard_DS4_v2" `
    -UpgradePolicyMode Automatic

$image = Get-AzureRMImage -ImageName $imageName -ResourceGroupName $ManagedImageResourceGroupName

Set-AzureRmVmssStorageProfile $vmssConfig `
   -OsDiskCreateOption FromImage `
   -ManagedDisk StandardLRS `
   -OsDiskCaching "None" `
   -OsDiskOsType Windows `
   -ImageReferenceId $image.id

After creating the VMSS you have a Scale Set with a single VM based on the custom image you’ve build with Packer. The next step is to install the VSTS Agent on the VMs and connect them to your VSTS account.

Installing the VSTS Agent

I’ve created a PowerShell script AddAgentToVM that takes your VSTS account and a Personal Access Token as parameters. It also asks for the user and credentials under which it should run the Agent. I’ve mapped these to the administrator account that’s also used to install all software on the VM.

To run the script on the newly created VM I use Add-AzureRmVmssExtension. This command lets you specify a script that’s stored in a Storage Account on Azure and that you want to run on your VMs. Uploading the AddAgentToVM to an Azure Storage Account is easy. I can then run it on the VM:


$publicSettings = @{
   "fileUris" = @("https://$StorageAccountName.blob.core.windows.net/$ContainerName/$FileName");
   "commandToExecute" = "PowerShell -ExecutionPolicy Unrestricted .$FileName -VSTSToken $VSTSToken -VSTSUrl $VSTSUrl -windowsLogonAccount $VMUser -windowsLogonPassword $VMUserPassword";
};

Add-AzureRmVmssExtension -VirtualMachineScaleSet $vmss `
   -Name "VSTS_Agent_Install" `
   -Publisher "Microsoft.Compute" `
   -Type "CustomScriptExtension" `
   -TypeHandlerVersion 1.8 `
   -ErrorAction Stop `
   -Setting $publicSettings

What’s next

And that’s it. You now have a build script that creates your custom image and a release script that creates a VM Scale Set and installs the VSTS agent on it. In the next blog post I’ll show you how to setup a Git repository and Build and Release definitions that fully automate building, deploying, scaling and managing your pool of Hosted Agents. The next part looks into building pipelines in VSTS to automatically build and deploy the agents.