Connecting to Data Lake Gen 2 from Azure Function using Service Principal is throwing AuthorizationPermissionMismatch error

kirtan_shah 0
Viewed 67

I am trying to connect to Data Lake Gen 2 account using Service Principal created to access Data Lake Gen 2 from an Azure Function ( Using Service Bus Topic Trigger ) This service principal is working fine with services like Azure Databricks. But when I try to connect from Azure function using the same service principal, it is throwing an AuthorizationPermissionMismatch error.

Only Service Principal or Managed Identity is allowed from Azure Function to access the data lake.

I am following the code mentioned in the below Microsoft documentation page:

https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet#connect-by-using-azure-active-directory-azure-ad

I was using the below Python example as a reference as I didn't get an end to end C# example:

https://docs.microsoft.com/en-us/azure/developer/python/tutorial-deploy-serverless-cloud-etl-05

The function code is as below:

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
using System.Threading.Tasks;
using Azure.Storage.Files.DataLake;
using Azure.Storage.Files.DataLake.Models;
using System.Collections.Generic;

namespace FunctionApp1
{
    public class Function1
    {


        [FunctionName("Function1")]
        public async Task RunAsync([ServiceBusTrigger("service_bus_name", "subscription_name", Connection = "shared_access_key_connection_name")] string mySbMsg, ILogger log)
        {

            string accountName = "";
            string clientID = "";
            string clientSecret = "";
            string tenantID = "";
            var credential = new ClientSecretCredential(
    tenantID, clientID, clientSecret, new TokenCredentialOptions());
            string dfsUri = "https://" + accountName + ".dfs.core.windows.net";


            DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClient(new Uri(dfsUri), credential);
            string adls_fsys_name = "";
            DataLakeFileSystemClient fileSystemClient = dataLakeServiceClient.GetFileSystemClient(adls_fsys_name);
            await ListFilesInDirectory(fileSystemClient);
            log.LogInformation($"C# ServiceBus topic trigger function processed message: {mySbMsg}");
        }

        public async Task ListFilesInDirectory(DataLakeFileSystemClient fileSystemClient)
        {
            string adls_dir_name = "";
            IAsyncEnumerator<PathItem> enumerator =
                fileSystemClient.GetPathsAsync(adls_dir_name).GetAsyncEnumerator();

            await enumerator.MoveNextAsync();

            PathItem item = enumerator.Current;

            while (item != null)
            {
                Console.WriteLine(item.Name);

                if (!await enumerator.MoveNextAsync())
                {
                    break;
                }

                item = enumerator.Current;
            }

        }
    }
}

The error that I got is :

2022-09-08T15:27:19.792 [Error] Executed 'Function1' (Failed, Id=c8d7cc56-f8df-4d51-b9ff-254dbe9d39b6, Duration=1166ms)This request is not authorized to perform this operation using this permission.RequestId:2b1dba4b-501f-00b8-2f97-c3d425000000Time:2022-09-08T15:27:19.7122855ZStatus: 403 (This request is not authorized to perform this operation using this permission.)ErrorCode: AuthorizationPermissionMismatchContent:{"error":{"code":"AuthorizationPermissionMismatch","message":"This request is not authorized to perform this operation using this permission.\nRequestId:2b1dba4b-501f-00b8-2f97-c3d425000000\nTime:2022-09-08T15:27:19.7122855Z"}}Headers:Server: Windows-Azure-HDFS/1.0,Microsoft-HTTPAPI/2.0x-ms-error-code: AuthorizationPermissionMismatchx-ms-request-id: 2b1dba4b-501f-00b8-2f97-c3d425000000x-ms-version: 2021-08-06x-ms-client-request-id: 7bfa87f0-16ab-44e3-b4b3-84a62b1222c8Date: Thu, 08 Sep 2022 15:27:19 GMTContent-Length: 227Content-Type: application/json; charset=utf-8
azure-data-lake azure-blob-storage .net-core azure-functions c#
1 Answers

Status: 403 (This request is not authorized to perform this operation using this permission.)ErrorCode: AuthorizationPermissionMismatchContent:{"error":{"code":"AuthorizationPermissionMismatch","message":"This request is not authorized to perform this operation using this permission.\nRequestId:2b1dba4b-501f-00b8-2f97-c3d425000000\nTime:2022-09-08T15:27:19.7122855Z"}}

The above 403 errors occurs you may not given proper permission to azure function and also you may not assign roles in storage data lake Gen 2 account .

For service principal authentication purpose you need to assign roles in your storage account.

  • Storage Blob Data Contributor - Write permission
  • Storage Blob Data Reader - Read permission

Azure portal ->Storage account-> Access Control (IAM)->Add role assignments->storage blob contributor role.

enter image description here

Check the firewall setting in the storage account:

enter image description here

In networking if you are access in public enable the select all network or if you enabled selected networks add the Virtual networks.

Reference: Copy and transform data in Azure Data Lake Storage Gen2 - Azure Data Factory & Azure Synapse | Microsoft Docs

Venkatesan0
Related