Spark shuffle fails with AccessDenied exception

I run Apache Spark stream job on cluster with: spark.master yarn spark.submit.deployMode cluster spark.shuffle.service.enabled true spark.dynamicAllocation.enabled true But it fails with Caused by: java.nio.file.AccessDeniedException: /tmp/hadoop-hadoop/nm-local-dir/usercache/hadoop/appcache/application_1590438937007_0020/blockmgr-ff867859-36d0-4db7-8243-bfabfb3bd40d/0d/shuffle_0_3_0.index. I run it as hadoop user. I cannot understand why process created that file ain't able to read (or modify) it