Skip to content

Hoppy7/adoLogStreamer

Repository files navigation

adoLogStreamer

Build Status

Overview

Azure DevOps currently offers no out-of-the-box functionality to stream pipeline logs to external endpoints and this sample can be used to fill in the gaps. This project streams ADO pipeline logs to Azure blob storage.


Log Stream Flow

image

  1. Templates/streamLogs.yml template stage gets the pipeline definition run's current context and puts it on the storage queue

  2. Function app triggers and processes message

  3. Function app calls the Azure DevOps Pipeline List Logs API and downloads the pipeline run's logs

  4. Function app pushes the pipeline run's logs to blob storage


Implementation

  1. Create or get your Azure DevOps Personal Access Token (PAT)

  2. Create a secret in Azure Key Vault named "adoLogStreamerPat" that will securely store the Azure DevOps PAT token. This is set as a keyvault reference in the function app's app settings in the arm template.

$pat = "<pat_token>"
$vaultName = "<keyvault_name>"
Set-AzKeyVaultSecret -VaultName $vaultName -Name "adoLogStreamerPat" -SecretValue $(ConvertTo-SecureString -AsPlainText $pat -Force)
  1. Update the pipeline's variables listed in the below table with your own values

    Variable Name Value Description
    azureSubscription The name of the service connection the pipeline will leverage to deploy the arm template
    resourceGroupName The name of the resource group the function app and storage account will be deployed
    location The region the Azure resource group and resources will be deployed
    functionAppName The name of the Azure function app. Note this must be a globally unique resource name
    azureDevOpsOrg The name of the Azure DevOps Organization
    azureDevOpsProject The name of the Azure DevOps Project
    adoPatKeyvaultSecretUri The uri of the secret created from #2 - https://{vaultName}.vault.azure.net/secrets/adoLogStreamerPat/
    storageAccountName The name of the storage account used by the function app to store logs. Note this must be a globally unique resource name
  2. Run the pipeline to deploy the arm template and the function app package

  3. Grant the function app's MSI access to the keyvault so it can access the adoPatKeyvaultSecretUri keyvault reference app setting

$functionAppName = "<functionApp_name>"
$vaultName = "<keyvault_name>"
$vaultRG = "<keyvault_resourceGroup>"
$msi = Get-AzADServicePrincipal -SearchString $functionAppName
Set-AzKeyVaultAccessPolicy -VaultName $vaultName -ResourceGroupName $vaultRG -ServicePrincipalName $msi.ApplicationId.Guid -PermissionsToSecrets "get, list"

image

About

Stream Azure DevOps pipeline logs to blob storage

Topics

Resources

License

Stars

Watchers

Forks