Susan Williams
December 03, 2015

Powershell, VSO Rest API and vNext Builds

Susan Williams

In this post I am going to be showing a simple example of using the Visual Studio Online REST API to retrieve information about a build using PowerShell. Then we'll take a look at how we can use the information we retrieved in a vNext build step. I am assuming at least some basic understanding of builds, Visual Studio Online and PowerShell.

vNext Builds

The new builds in VSO allow for easy customization of the build pipeline. You can add and rearrange steps to create a process that works best for your application. For this post I will be adding a few steps for demonstration but will not be going into much detail on all of them.

First, I created a simple console application and added it to a Git repository in my VSO account (you can get one for free at http://www.visualstudio.com). The only code I added to the Main method is a simple try/catch that will come into play later.

<code>
try  {
           Console.Write("Hello, world!");            
       }             
catch (Exception ex)            
      {                 
           Console.Write("There was an error!");             
      }
</code>

In VSO go to the Build tab and select the green plus symbol to add a new build definition. Start with an Empty definition.

VSO Build Tab

After selecting 'OK' you will be taken to the Completed builds screen which is, of course, empty because we have not queued a build. It defaulted the name of the new definition to "New Empty definition". Select the Edit link next to the name. On this screen you will see many options for customizing your build.

Select the Repository tab and select the repository and branch you will use for the build. Mine is a Git repository, the name is BuildPipeline and I want to use the master branch.

Repository Build Pipeline

Hit the Save button and you will be prompted to change the name and enter a comment. You will be prompted with this on every save and selecting the History navigation item you can view all of the changes made to the build definition.

Back in the Build tab let's add our first build step (On the modal that comes up it says "Add Tasks" - task and step are used interchangeably). Select Add next to "Visual Studio Build". It will add the step but you will need to close the modal to return to the Build screen.

Build Tab Add Next

Set the name for this step to "Compile" and browse for the solution of your project you want built in this step.

Compile

Save your updates and then queue a build.

One last thing to note - select the Variables navigation item and copy the Value for the system.definitionId somewhere. We will need this later.

Now we'll step out of VSO for a while and take a look at the REST API.

Visual Studio Online REST API & PowerShell

Last year Microsoft released a new API for accessing Visual Studio Online. Using the REST API you can access pretty much anything in VSO - task boards, git commits, project and teams. You can review all of the services available here -https://www.visualstudio.com/en-us/integrate/api/overview.

This post will be focusing on the service for Builds using version 2.0 of the API.

The first URL we will look at is the list of all builds for the build definition we created previously (https://www.visualstudio.com/integrate/api/build/builds#Getalistofbuilds). The parameters I will use are definitions and $top (the api-version parameter is required in all calls). So our URL will look like this:

<code>
https://{account}.visualstudio.com/defaultcollection/{project}/_apis/build/builds?api-version{version}&definitions={definitionId}&$top={int}
</code>

In our PowerShell script we can create a handful of variables to help us out.

<code>
$projectName = "" #Project name in VSO - can be found in the top left header when looking at builds
$account = "" #Your account name 
$username = "" #Alternate credentials username
$password = "" #Alternate credentials password
$definition = "" #definition Id(s) found in the Variables tab of a build definition
$apiVersion = "2.0"
$tfsUrl = 'https://' + $account + '.visualstudio.com/defaultcollection/' + $projectName   #Base url for all of the VSO API calls 
</code>

Accessing the VSO API through PowerShell requires Alternate Credentials set up. I found the following post helpful for the syntax on the headers needed for credentials. It also has some links to information on Alternate Credentials if you do not have them set up. http://stuartpreston.net/2014/05/accessing-visual-studio-online-rest-api-using-powershell-4-0-invoke-restmethod-and-alternate-credentials/.

<code>
$basicAuth = ("{0}:{1}" -f $username,$password)
$basicAuth = [System.Text.Encoding]::UTF8.GetBytes($basicAuth)
$basicAuth = [System.Convert]::ToBase64String($basicAuth)
$headers = @{Authorization=("Basic {0}" -f $basicAuth)}
</code>

Now we'll put some of the variables together and create the full url to get the most recent build and call it. For the $top parameter I am specifying 1 to only get the most recent build queued.

<code>
[uri] $uri = $tfsUrl + "/_apis/build/builds?api-version=" + $apiVersion + "&definitions=" + $definition + "&`$top=1"
$allBuildDefs = Invoke-RestMethod -Uri $uri -Headers $headers -Method Get
</code>

To review the Json that gets returned, send it out to a text file.

<code>
$allbuildDefs | ConvertTo-Json | Out-File -FilePath ‘d:\topBuildDefinition.txt’ -Force
</code>

There is some interesting information about the build in this data but what we are looking for is the buildNumber so we can start digging deeper into the build details.

The next call to the VSO API we will look at is the Build Details (https://www.visualstudio.com/integrate/api/build/builds#Getbuilddetails).

<code>
https://{account}.visualstudio.com/defaultcollection/{project}/_apis/build/builds/{buildid}/timeline?api-version={version}
</code>

Notice in the URL it ends with the build id and /timeline. This will not only get us general details about the build but also each of the steps that were completed in the build definition.

We will save the buildNumber to a variable and use it in building out the next URL. The build number is found inside the value array.

<code>
$buildNumber = ($allbuildDefs.value).buildNumber
[uri] $uri = $tfsUrl + "/_apis/build/builds/" + $buildNumber + "/timeline?api-version=" + $apiVersion
$currentBuild = (Invoke-RestMethod -Uri $uri -Headers $headers -Method Get)
$currentBuild | ConvertTo-Json | Out-File -FilePath ‘d:\buildDetails.txt’ -Force
</code>

This gets us even more interesting information about the build. We can see the status of each task/step and the progress of it, the number of warnings from the task and even urls to the log files that were generated.

Now we can take this information and start using it to customize our build process even more. For example, if there are certain steps that we want to make sure don't exceed a set number of warnings, we can fail the build.

Let's try it out.

PowerShell and vNext Builds

We will need to make a few changes to our script before we can use it in the build definition. First, the script should be reusable so we will want to add a couple of parameters, one for the specific step to look at and the maximum number of warnings allowed.

<code>
Param(
[string]$maxWarnings,
[string]$taskName)
</code>

Next, instead of having to get the most recent build (which would get the one executing the script) VSO has some environment variables that can be used (https://msdn.microsoft.com/en-us/library/hh850448.aspx) and one is for the build number of the currently running build. It is very helpful to use these environment variables because it results in less code and one less call to the API.

<code>
$buildNumber = $env:BUILD_BUILDNUMBER
</code>

Now let's get the number of warnings that were generated for the specified Task in the current build. In the JSON returned for the current build it has a record object that holds an array of the tasks. We will select the object in the array where the Name value equals the TaskName parameter. From that object we want to get the warningCount value

<code>
$warningCount = $currentBuild.records | Where { $_.name -eq $TaskName } | select warningCount
</code>

We also want to make sure to fail the build if the number of warnings in the step we are looking at is more than the max number of warnings that were passed in. A simple If statement with a descriptive error can take care of this. To fail the build we just exit the script with a status that is not 0.

<code>
IF ($warningCount.warningCount -gt $maxWarnings)
{
    Write-Error("The number of  warnings (" + $warningCount.warningCount + ") exceeds the number of allowed warnings (" + $maxWarnings + ")")
    exit 1
}
</code>

Below is the full script. I also added in a couple Write-Verbose statements to print out the number of warnings and the max number of warnings that was passed in to the console.

<code>
Param( 
[string]$maxWarnings, 
[string]$taskName )
$account = "YourAccountName"
$username = "YourUsername"
$password = "YourPassword"
$projectName ="YourProjectName"
$definition = YourDefinitionId
$apiVersion = "2.0"
$buildNumber = $env:BUILD_BUILDNUMBER
$tfsUrl = 'https://' + $account + '.visualstudio.com/defaultcollection/' + $projectName
$basicAuth = ("{0}:{1}" -f $username,$password)
$basicAuth = [System.Text.Encoding]::UTF8.GetBytes($basicAuth)
$basicAuth = [System.Convert]::ToBase64String($basicAuth)
$headers = @{Authorization=("Basic {0}" -f $basicAuth)}
[uri] $uri = $tfsUrl + "/_apis/build/builds/" + $buildNumber + "/timeline?api-version=" + $apiVersion
$currentBuild = (Invoke-RestMethod -Uri $uri -Headers $headers -Method Get)
$warningCount = $currentBuild.records | Where { $_.name -eq $TaskName } | select warningCount
Write-Verbose ("Warnings: " + $warningCount.warningCount) -Verbose
Write-Verbose ("Max warnings allowed: " + $maxWarnings) -Verbose
IF ($warningCount.warningCount -gt $maxWarnings)
{
    Write-Error("The number of  warnings (" + $warningCount.warningCount + ") exceeds the number of allowed warnings (" + $maxWarnings + ")")
    exit 1
}
</code>

Save the PowerShell script and add it to your Visual Studio project. I added a new PowerShell folder to the file structure in File Explorer to hold the script. In my solution, I added a new Solution Folder and then added the script using Add > Existing Item. Make sure to sync the changes to the project's Git repository.

PowerShell Folder

Back in VSO lets add a new step to the build definition we created earlier. This time go to the Utility group and add PowerShell. Make sure it comes after the Compile step already there and give it a more meaningful name. I gave it the name Compile Warning Check. For the Script file name, selecting the browse (…) button will allow you to select a script from your repository. Navigate to the PowerShell script created earlier and select OK.

Utility Group

In the Arguments text box enter the arguments to be passed in to the script. These will be standard argument formatting. I am using named parameters.

<code>
 -maxWarnings "1" -TaskName "Compile" 
</code>

Save the build definition and let's see it in action. On the left side of the Build page right-click your build and select Queue Build.

The new builds have a very useful console display that you can watch the output on. It shows when a task is starting and when it has finished. On the left hand side it has icons indicating which step is running and if they passed.

Build Success

Here you notice that the build succeeded and the two lines we wrote from the PowerShell script have been written out as well. The number of warnings (also seen higher in the console written in yellow) does not exceed the max number of warnings we specified.

Head back to the build definition and edit the max number of warnings to be 0 (we have very high expectations for this project). Save and queue another build.

Build Failure

Getting this to run was the first time I was excited to successfully fail a build.

Only the beginning

In this post we took a look at three different tools and services that are each useful to know in depth on their own. Instead of diving into one topic we took a little bit of functionality from each and created something that in itself can be very customizable for different situations.

6 Comments

  1. 6 Chris 04 Dec
    Thanks for the insights. Very clear and concise breakdown of how to approach. 
  2. 5 Lars Ole Christiansen 15 Feb
    Could you show or send to me where it will look if i like one local TFS server .... http: // tfsxxxx: 8080 / tfs / collecention
  3. 4 Susan Williams 15 Feb
    Hi Lars,
    I do not have experience with on premise TFS but looking at the documentation for the REST API here, https://www.visualstudio.com/integrate/get-started/rest/basics, you should be able to use the same calls. 

    Instead of the following that I have used in the blog:  
    https://{account}.visualstudio.com/defaultcollection/{projectname}

    You should be able to use:
    https://{server}:8080/defaultcollection/
    So the example you included in your comment would be:
    https://tfsxxxx:8080/defaultcollection/{projectname}

    You will still need to enable alternate credentials (http://stuartpreston.net/2014/05/accessing-visual-studio-online-rest-api-using-powershell-4-0-invoke-restmethod-and-alternate-credentials/) to use basic authentication.
  4. 3 Nilesh 20 Jun
    Above solution works fine for TFS online service, for on promises use below script to update variable

    NOTE : To consume REST API enable basic authentication on tfs site

    [String]$buildID = "$env:BUILD_BUILDID"
    [String]$project = "$env:SYSTEM_TEAMPROJECT"
    [String]$projecturi = "$env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI"

    $username="username"
    $password="password"
    $basicAuth= ("{0}:{1}"-f $username,$password)
    $basicAuth=[System.Text.Encoding]::UTF8.GetBytes($basicAuth)
    $basicAuth=[System.Convert]::ToBase64String($basicAuth)
    $headers= @{Authorization=("Basic {0}"-f $basicAuth)}

    $buildurl= $projecturi + $project + "/_apis/build/builds/" + $buildID + "?api-version=2.0"

    Write-Host "Getting build from url: "$buildurl
    $getbuild = Invoke-RestMethod -Uri $buildurl -headers $headers -Method Get |select definition

    $definitionid = $getbuild.definition.id
    Write-Host "Defination Id "$definitionid

    $defurl = $projecturi + $project + "/_apis/build/definitions/" + $definitionid + "?api-version=2.0"

    Write-Host "Getting defination from url: "$defurl
    $definition = Invoke-RestMethod -Uri $defurl -headers $headers -Method Get

    $definition.variables.VariableName.value = "Value to update"

    $json = @($definition) | ConvertTo-Json  -Depth 999

    $updatedef = Invoke-RestMethod  -Uri $defurl -headers $headers -Method Put -Body $json -ContentType "application/json"

    Hope it helps for on promises tfs #happy_deving #happy_coding
  5. 2 Surabhi 08 Nov
    Can I create a variable in my Powershell script and then use it in my next task in vNext Build?
  6. 1 Susan Williams 10 Nov
    Hi Surabhi,

    Yes, you can create a variable in one step and then access it in another task.

    To set up a simple proof of concept:
    1. Create an empty build definition
    2. Add two PowerShell Script tasks
    3. In the first PowerShell task set the Type to "Inline Script" and paste in the following script. Replace "samplevariable" with the name of the variable and "testvalue" with the value you want it set to.
           Write-Host "##vso[task.setvariable variable=samplevariable;]testvalue"
    4. In the second PowerShell task set the Type to "Inline Script" and paste in the following script. Replace "samplevariable" with the name of the variable you set in step 3.
           Write-Host $env:samplevariable
    5. Save and queue a new build.  In the console while it is running, or looking at the log for the second Powershell script created you will see the value set in step 3 printed on the screen.

    The script in step 3 creates a new variable for that build so you will be able to access it in any task type after it has been created. You can also use the script to change the value of a variable that was created in the Variables tab when editing a build definition.

    See this page for various Logging Commands: https://github.com/Microsoft/vsts-tasks/blob/master/docs/authoring/commands.md
    See this page for more information on build variables: https://www.visualstudio.com/en-us/docs/build/define/variables

Comment

  1. RadEditor - HTML WYSIWYG Editor. MS Word-like content editing experience thanks to a rich set of formatting tools, dropdowns, dialogs, system modules and built-in spell-check.
    RadEditor's components - toolbar, content area, modes and modules
       
    Toolbar's wrapper 
     
    Content area wrapper
    RadEditor's bottom area: Design, Html and Preview modes, Statistics module and resize handle.
    It contains RadEditor's Modes/views (HTML, Design and Preview), Statistics and Resizer
    Editor Mode buttonsStatistics moduleEditor resizer
      
    RadEditor's Modules - special tools used to provide extra information such as Tag Inspector, Real Time HTML Viewer, Tag Properties and other.
       

Subscribe to Our Blog

Get the latest blog posts sent right to your inbox so you never miss an informative post from Mercury.

 

Get Your Share On!

 

Tags