Transfer Azure batch windows container task output file to batch task working directory fails

Viewed 21

I am prototyping an azure batch sample using windows container workload in batch task. This container expects an input file, executes an algorithm and writes an output file st01_demo_results.csv inside the container at path C:\app\output (this directory is created in dockerfile build). On batch task success, the output file will be uploaded to azure storage container using SAS URL. I have mostly used this sample to build my prototype, just that instead of using exe, I am using windows container. I am trying to transfer my output file generated inside windows container to batch task working directory by mounting a volume with source as batch task working directory and target my container output file path.

string taskCommandLine = $"{inputMediaFile} C:\\app\\output\\{outputMediaFile}";
string containerRunOptions = "-v %AZ_BATCH_TASK_WORKING_DIR%:c:\\app\\output:rw";
    var task = new CloudTask(taskId, taskCommandLine)
    {
      UserIdentity = new UserIdentity(autoUserSpec),
      ContainerSettings = containerSettings,
    };
    task.ResourceFiles = new List<ResourceFile> { inputFiles[i] };

With this, I get exception Unhandled exception. System.UnauthorizedAccessException: Access to the path 'c:\app\output\st01_demo_results.csv' is denied.

If, I don't provide volume binding in containerRunOptions then my container task executes fine but then I don't get the file transferred to my batch task working directory. I suspect, that azure batch somewhere ignores rw option and forcefully makes the volume binding read-only - because when I tried running the same windows container locally in docker desktop with ro option, I get same unauthorized exception.

1 Answers

You need to ensure that c:\\app\\output is accessible by the container user that is currently executing within the container context, typically ContainerUser, ContainerAdministrator, or some custom user that you may have added as part of your DOCKERFILE.

Related