Azure Data Factory Delete Activity - Will not delete folder

ADF has Storage Blob Data Contributor access which gives write permissions. I have the delete activity setup the same as the Microsoft guide below (second example)

• No file name specified
• Recursive = true But only the files within the folder are deleted and not the folder.

I tried running the delete activity on an empty folder as well and it did not work. It will first succeed and say that it has deleted the 1 file (i.e. the empty folder). Then I run the pipeline again and it will say the blob cannot be found. So ADF does sort of think that its deleting the folder, but the folder is still there in storage explorer.

Any advice? Thanks!

{
    "name": "pipeline1_delete",
    "properties": {
        "activities": [
            {
                "name": "Delete1",
                "type": "Delete",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "dataset": {
                        "referenceName": "DS_BinaryDeleteFolder",
                        "type": "DatasetReference"
                    },
                    "enableLogging": false,
                    "storeSettings": {
                        "type": "AzureBlobStorageReadSettings",
                        "recursive": true,
                        "enablePartitionDiscovery": false
                    }
                }
            }
        ],
        "annotations": []
    }
}