Categories
Azure

ADO Pipeline library export and import

In the Azure DevOps world, moving from one tenant to another is not a very familiar concept based on Microsoft documentation. I had to do exactly this. The Azure DevOps Migration Tools did the trick for most of the ADO Boards-related things.

Repos were copied over by pushing to a new Git source.

Pipelines are stored in the Git repository as YAML and were imported / linked again.

One large task was migrating the pipeline library. You can’t export the values secrets in the library, but I used a tool to make sure to copy everything over. Having the secret variables copied over with empty values was already very valuable to me.

ADO Library Export

Running the export will save the library as a JSON file on your computer. You also have the option to modify it with string replacement if needed. Once ready, it will be imported again with the second script.

# web-performance.ch 2025 ADO Library Export

function Export-AzureDevOpsVariableGroups {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true)]
        [string]$OrganizationName,
        
        [Parameter(Mandatory = $true)]
        [string]$ProjectName,
        
        [Parameter(Mandatory = $true)]
        [string]$PersonalAccessToken,
        
        [Parameter(Mandatory = $true)]
        [string]$OutputPath,
        
        [Parameter(Mandatory = $false)]
        [string]$ApiVersion = "7.1-preview.1"
    )
    
    try {
        # Validate parameters
        if ([string]::IsNullOrWhiteSpace($OrganizationName)) { throw "Organization name cannot be empty" }
        if ([string]::IsNullOrWhiteSpace($ProjectName)) { throw "Project name cannot be empty" }
        if ([string]::IsNullOrWhiteSpace($PersonalAccessToken)) { throw "Personal Access Token cannot be empty" }

        # Encode PAT for Authorization header
        $base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($PersonalAccessToken)"))
        
        # Set API endpoint
        $apiUrl = "https://dev.azure.com/$OrganizationName/$ProjectName/_apis/distributedtask/variablegroups?api-version=$ApiVersion"
        
        # Set up headers
        $headers = @{
            Authorization = "Basic $base64AuthInfo"
            'Content-Type' = 'application/json'
        }

        # Send API request
        $response = Invoke-RestMethod -Uri $apiUrl -Method Get -Headers $headers -ErrorAction Stop

        if ($response.value) {
            # Create export data structure
            $exportData = $response.value | ForEach-Object {
                $variables = @{}
                foreach ($var in $_.variables.PSObject.Properties) {
                    $variables[$var.Name] = @{
                        value = $var.Value.value
                        isSecret = $var.Value.isSecret
                    }
                }

                @{
                    name = $_.name
                    description = $_.description
                    variables = $variables
                }
            }

            # Save to JSON file
            $exportData | ConvertTo-Json -Depth 10 | Out-File -FilePath $OutputPath -Encoding UTF8
            Write-Host "`nExported $($response.value.Count) variable groups to $OutputPath" -ForegroundColor Green
        } else {
            Write-Warning "No variable groups found to export."
        }
    }
    catch [System.Net.WebException] {
        $statusCode = [int]$_.Exception.Response.StatusCode
        switch ($statusCode) {
            401 { Write-Error "Authentication failed. Check your Personal Access Token." }
            403 { Write-Error "Access denied. Check your permissions." }
            404 { Write-Error "Project or Organization not found." }
            default { Write-Error "HTTP Error $($statusCode): $($_.Exception.Response.StatusDescription)" }
        }
    }
    catch {
        Write-Error "An unexpected error occurred: $($_.Exception.Message)"
    }
}



# Example usage:
# Export variable groups
Export-AzureDevOpsVariableGroups `
    -OrganizationName "FIRST" `
    -ProjectName "TestProject" `
    -PersonalAccessToken "" `
    -OutputPath "D:\vargroups\origin-variable_groups.json"

ADO Library Import

Running the import will use the JSON file to create the same library groups and also add the values where they were present.

# web-performance.ch 2025 ADO Library Import

function Import-AzureDevOpsVariableGroups {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true)]
        [string]$OrganizationName,
        
        [Parameter(Mandatory = $true)]
        [string]$ProjectName,
        
        [Parameter(Mandatory = $true)]
        [string]$PersonalAccessToken,
        
        [Parameter(Mandatory = $true)]
        [string]$InputPath,
        
        [Parameter(Mandatory = $false)]
        [string]$ApiVersion = "7.1-preview.1",

        [Parameter(Mandatory = $false)]
        [switch]$UpdateExisting,

        [Parameter(Mandatory = $false)]
        [switch]$Verbose2
    )
    
    try {
        # Validate parameters and file
        if (!(Test-Path $InputPath)) { throw "Input file not found: $InputPath" }

        # Read and parse JSON file
        $importData = Get-Content $InputPath -Raw | ConvertFrom-Json
        
        # Encode PAT for Authorization header
        $base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($PersonalAccessToken)"))
        
        # Set up headers
        $headers = @{
            Authorization = "Basic $base64AuthInfo"
            'Content-Type' = 'application/json'
        }

        # Base API URL
        $baseUrl = "https://dev.azure.com/$OrganizationName/$ProjectName/_apis/distributedtask/variablegroups"

        foreach ($group in $importData) {
            Write-Host "`nProcessing group: $($group.name)" -ForegroundColor Cyan
            
            # Prepare variables object
            $variables = @{}
            foreach ($var in $group.variables.PSObject.Properties) {
                $variables[$var.Name] = @{
                    value = $var.Value.value
                    isSecret = if ($var.Value.isSecret -eq $true) { $true } else { $false }
                }
            }

            # Prepare request body
            $body = @{
                name = $group.name
                description = $group.description
                variables = $variables
                type = "Vsts"
                variableGroupProjectReferences = @(
                    @{
                        name = $group.name
                        description = $group.description
                        projectReference = @{
                            id = $null  # Will be populated by Azure DevOps
                            name = $ProjectName
                        }
                    }
                )
            }

            if ($Verbose2) {
                Write-Host "Request body:" -ForegroundColor Yellow
                Write-Host ($body | ConvertTo-Json -Depth 10)
            }

            $bodyJson = $body | ConvertTo-Json -Depth 10

            if ($UpdateExisting) {
                # Check if group exists
                Write-Host "Checking for existing group..." -ForegroundColor Gray
                $existingGroups = Invoke-RestMethod -Uri "$baseUrl`?api-version=$ApiVersion" -Method Get -Headers $headers
                $existingGroup = $existingGroups.value | Where-Object { $_.name -eq $group.name }

                if ($existingGroup) {
                    Write-Host "Updating existing group: $($group.name)" -ForegroundColor Yellow
                    $updateUrl = "$baseUrl/$($existingGroup.id)?api-version=$ApiVersion"
                    try {
                        $result = Invoke-RestMethod -Uri $updateUrl -Method Put -Headers $headers -Body $bodyJson
                        Write-Host "Successfully updated group: $($group.name)" -ForegroundColor Green
                    }
                    catch {
                        $errorResponse = $_.ErrorDetails.Message
                        Write-Host "Error updating group. Response: $errorResponse" -ForegroundColor Red
                        throw
                    }
                } else {
                    Write-Host "Creating new group: $($group.name)" -ForegroundColor Green
                    try {
                        $result = Invoke-RestMethod -Uri "$baseUrl`?api-version=$ApiVersion" -Method Post -Headers $headers -Body $bodyJson
                        Write-Host "Successfully created group: $($group.name)" -ForegroundColor Green
                    }
                    catch {
                        $errorResponse = $_.ErrorDetails.Message
                        Write-Host "Error creating group. Response: $errorResponse" -ForegroundColor Red
                        throw
                    }
                }
            } else {
                Write-Host "Creating group: $($group.name)" -ForegroundColor Green
                try {
                    $result = Invoke-RestMethod -Uri "$baseUrl`?api-version=$ApiVersion" -Method Post -Headers $headers -Body $bodyJson
                    Write-Host "Successfully created group: $($group.name)" -ForegroundColor Green
                }
                catch {
                    $errorResponse = $_.ErrorDetails.Message
                    Write-Host "Error creating group. Response: $errorResponse" -ForegroundColor Red
                    throw
                }
            }
        }

        Write-Host "`nImport completed successfully!" -ForegroundColor Green
    }
    catch [System.Net.WebException] {
        $errorResponse = $null
        if ($_.Exception.Response) {
            $stream = $_.Exception.Response.GetResponseStream()
            $reader = New-Object System.IO.StreamReader($stream)
            $errorResponse = $reader.ReadToEnd()
            Write-Host "Error Response: $errorResponse" -ForegroundColor Red
        }
        
        $statusCode = [int]$_.Exception.Response.StatusCode
        switch ($statusCode) {
            400 { Write-Error "Bad Request (400): The request was invalid or malformed. Details: $errorResponse" }
            401 { Write-Error "Authentication failed (401). Check your Personal Access Token." }
            403 { Write-Error "Access denied (403). Check your permissions." }
            404 { Write-Error "Not found (404). Project or Organization not found." }
            default { Write-Error "HTTP Error $($statusCode): $($_.Exception.Response.StatusDescription). Details: $errorResponse" }
        }
    }
    catch {
        Write-Error "An unexpected error occurred: $($_.Exception.Message)"
        if ($_.ErrorDetails) {
            Write-Host "Error Details: $($_.ErrorDetails.Message)" -ForegroundColor Red
        }
        if ($_.Exception.Response) {
            $reader = New-Object System.IO.StreamReader($_.Exception.Response.GetResponseStream())
            $errorResponse = $reader.ReadToEnd()
            Write-Host "Full Response: $errorResponse" -ForegroundColor Red
        }
    }
}

# Example usage with verbose output:
Import-AzureDevOpsVariableGroups `
    -OrganizationName "SECOND" `
    -ProjectName "TestProject" `
    -PersonalAccessToken "" `
    -InputPath "D:\vargroups\updated-variable_groups.json" `
    -UpdateExisting 
    -Verbose2

The scripts were massively valuable in my migration needs in November 2024. They are provided here free of charge as is with no warranty or further documentation. If you plan to use them, familiarize yourself first and maybe test on an unimportant tenant.