Databricks API 2.0 - create secret scope in powershell using service principal credentials

I am trying to create a key vault backed secret scope in Azure databricks using a powershell script that runs during Azure DevOps deployment. It works fine when I run locally using my own credentials but I get an error when I try to run it using the service principal credentials.

The problem I'm having is similar to but not exactly the same as this previous post.

Here is my script:

[CmdletBinding()]
Param(
    $azureADDatabricksAccessToken = $env:AZUREADDATABRICKSACCESSTOKEN,
    $azureManagementAccessToken = $env:AZUREMANAGEMENTACCESSTOKEN,
    $workspaceResourceId,
    $subscription,
    $resourceGroup,
    $keyVault,
    $workspaceUrl,
    $scope
)
$headers = @{ 
"Authorization" = "Bearer $azureADDatabricksAccessToken";
"X-Databricks-Azure-SP-Management-Token" = $azureManagementAccessToken;
"X-Databricks-Azure-Workspace-Resource-Id" = $workspaceResourceId;
}

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$scopes = (Invoke-RestMethod -Uri "https://$workspaceUrl/api/2.0/secrets/scopes/list" -Method Get -Headers $headers).scopes
$exists = ($scopes | Where-Object {$_.name -eq $scope}).Count -gt 0
if($exists){
    Write-Host "Secret scope found";
}
else{
    Write-Host "Creating new secret scope";
    
    $body = @{
     "scope" = "$scope";
     "scope_backend_type" = "AZURE_KEYVAULT";
     "backend_azure_keyvault" =
     @{
       "resource_id" = "/subscriptions/$subscription/resourceGroups/$resourceGroup/providers/Microsoft.KeyVault/vaults/$keyVault";
       "dns_name" = "https://$keyVault.vault.azure.net/";
     };
     "initial_manage_principal" = "users";
    }

    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
    Invoke-RestMethod -Uri "https://$workspaceUrl/api/2.0/secrets/scopes/create" -Method Post -Headers $headers -Body (ConvertTo-Json $body)
}

I get my access tokens like this:

$azureADDatabricksAccessToken = (az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --resource-type aad-graph | ConvertFrom-Json).accessToken
$azureManagementAccessToken = (az account get-access-token --resource "https://management.core.windows.net/" | ConvertFrom-Json).accessToken

This works when I log in using az login -t XXXX but it fails when run as a service principal using az login --service-principal -u XXXX -p XXXX --tenant XXXX.

The error message I get is:

error_code":"CUSTOMER_UNAUTHORIZED","message":"Unable to grant read/list permission to Databricks service principal to KeyVault 
'XXXXX': key not found: https://graph.windows.net/

Is there some other access token header I need to add for graph.windows.net when running as a service principal?