I have a the following bicep script to perform the following steps:
- Create storage accounts based on code
- create a key
- encrypt storage account with the correct key
- generate a secret connection string (storage account) into a key vault
The infra is splitted into 2 resource groups:
rg-shared => where the 2 key vault are (1 keyvault for key and GUID, and 1 keyvault for secrets (connection string)
rg-storage-account => where all the storage account get created
In azure bicep I have the following scripts:
Storage.bicep
param ManagedIdentityid string
param uri string
param kvname string
param keyvaultrg string = 'XXXX' //<== SHARED Resource Group
var keyVaultKeyPrefix = 'Key-Data-'
var storagePrefix = 'sthritesteur'
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = [for tenantCode in tenantCodes: {
name: '${storagePrefix}${tenantCode}'
location: resourceGroup().location
kind: 'StorageV2'
sku: {
name: 'Standard_RAGRS'
}
// Assign the identity
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${ManagedIdentityid}':{}
}
}
properties: {
allowCrossTenantReplication: true
minimumTlsVersion: 'TLS1_2'
allowBlobPublicAccess: false
allowSharedKeyAccess: true
networkAcls: {
bypass: 'AzureServices'
virtualNetworkRules: []
ipRules: []
defaultAction: 'Allow'
}
supportsHttpsTrafficOnly: true
encryption: {
identity: {
// specify which identity to use
userAssignedIdentity: ManagedIdentityid
}
keySource: 'Microsoft.Keyvault'
keyvaultproperties: {
keyname: '${kvname}-${keyVaultKeyPrefix}${toUpper(tenantCode)}'
keyvaulturi:uri
}
services: {
file: {
keyType: 'Account'
enabled: true
}
blob: {
keyType: 'Account'
enabled: true
}
}
}
accessTier: 'Hot'
}
}]
resource storage_Accounts_name_default 'Microsoft.Storage/storageAccounts/blobServices@2021-04-01' = [ for (storageName, i) in tenantCodes :{
parent: storageAccount[i]
name: 'default'
properties: {
changeFeed: {
enabled: false
}
restorePolicy: {
enabled: false
}
containerDeleteRetentionPolicy: {
enabled: true
days: 7
}
cors: {
corsRules: []
}
deleteRetentionPolicy: {
enabled: true
days: 30
}
isVersioningEnabled: true
}
}]
module connectionString 'shared.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(keyvaultrg)
name: storageName
params: {
storageAccountString: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
}
}]
keyvaultclient.bicep
param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo
param storagerg string = 'XXXX' //<=== Storage Accounts Resource Groups
param sharedManagedIdentity string = 'mgn-identity-shared'
param keyvaultmain string = 'XXXX' //<=== KeyVault Name where to create GUID AND Keys
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
var clientDataKeyPrefix = 'Key-Data-'
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
name: sharedManagedIdentity
location: resourceGroup().location
}
resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
name: keyvaultmain
}
resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
parent:keyVaultClients
name: '${keyVaultClients.name}-${clientDataKeyPrefix}${toUpper(code)}'
properties: {
keySize: 2048
kty: 'RSA'
// Assign the least permission
keyOps: [
'unwrapKey'
'wrapKey'
]
}
}]
resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2021-06-01-preview' = {
parent:keyVaultClients
name: 'add'
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: managedIdentity.properties.principalId
permissions: {
// minimum required permission
keys: [
'get'
'unwrapKey'
'wrapKey'
]
}
}
]
}
}
resource clientLearnersGuid 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultClients
name: '${keyVaultClients.name}${tenant}'
properties: {
contentType: 'GUID Key'
value: '${deploymentIdOne}-${deploymentIdTwo}'
}
dependsOn:kvClientsKey
}]
module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(storagerg)
name: storageName
params: {
ManagedIdentityid:managedIdentity.id
kvname:keyVaultClients.name
uri:keyVaultClients.properties.vaultUri
}
dependsOn:clientLearnersGuid
}]
shared.bicep
param keyvaultshared string = 'XXXX' //<=== Key Vault Where to Store the Storage Connection String Secret
param storageAccountString string
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
resource keyVaultShared 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
name: keyvaultshared
}
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultShared
name: '${keyVaultShared.name}-test${tenant}'
properties:{
contentType: '${tenant} Storage Account Connection String'
value: storageAccountString
}
}]
Those script performs all the steps I needed based on the tenantCode. Which is just perfect. If I have only 1 tenantCode declared, everything goes smooth and perfect, but the issue I am facing raises when I try to declare more than 1. And this is the problem in details.
When I declare more than 1 code, the script still created all the resources I needed: Storage accounts,Keys,Encryptions, GUID and ConnectionStrings Secrets. But it fails anyway on the ConnectionStrings Secret.
The reason why it fails it because in those files at this block of code:
Shared.bicep
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultShared
name: '${keyVaultShared.name}-test${tenant}'
properties:{
contentType: '${tenant} Storage Account Connection String'
value: storageAccountString
}
}]
and
keuvaultsclient.bicep*
module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(storagerg)
name: storageName
params: {
ManagedIdentityid:managedIdentity.id
kvname:keyVaultClients.name
uri:keyVaultClients.properties.vaultUri
}
dependsOn:clientLearnersGuid
}]
I have a multiple loop, on which I realised that in my shared keyvault under secrets, I have the correct amount of secrets (with 3 tenant codes I have 3 secrets) and under each secret I have 3 versions (for each tenant code it generate a new version for each secret). This looping error causes bicep script to fail with the following message:
{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n }\\r\\n}\"\r\n },\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"}]}}
I am totally blocked at this stage and as I am a bicep beginner, I run out of all options and test to try to solve this issue.
How to reproduce
- Create 2 resource groups(one for the key vaults and one for the storage accounts)
- Update parameters with the correct kv names
- In all files add atleast 2 tenantCode on each file
- execute the command
az deployment group create -f ./keyvaultsclient.bicep -g <rg-where-keyvaults-are>
I hope I explained clearly enough the issue I am facing and please if you need more details just let me know.
Thank you so much for your time and help
