Angular Build EPERM: operation not permitted, copyfile

I'm running an Angular 12 build in Jenkins in a Docker container and am seeing this error when trying to copy asset files to the dist folder:

✖ Copying of assets failed.
Unable to copy assets: EPERM: operation not permitted, copyfile '/var/jenkins_home/workspace/MY-Web-App/src/assets/building.svg' -> '/var/jenkins_home/workspace/MY-Web-App/dist/my-web-app/assets/building.svg'

The problem started after we updated all the OS packages on the Ubuntu docker host, on June 15th, 2021.

First thought was the seccomp is wrong, but I have this in the docker-compose:

  jenkins:
    build: ./jenkins
    image: my-ci-jenkins
    hostname: ....
    user: root
    volumes:
      - /data/ci/jenkins/jenkins_home:/var/jenkins_home
      - artifactsvolume:/var/jenkins_home/workspace/artifacts/dist/my-web-app
    ports:
      - '8080:8080'
      - '50000:50000'
    security_opt:
      - seccomp:./jenkins/seccomp.json
    depends_on:
      - mariadb
      - gitea
    environment:
      - TZ=America/Los_Angeles
      - JENKINS_HOME=/var/jenkins_home
    restart: always
    networks:
      my-ci-net:
        ipv4_address: ....

and this, among other things, in the seccomp.json:

{
    "defaultAction": "SCMP_ACT_ERRNO",
    "syscalls": [
        ...
        {
            "name": "copyfile",
            "action": "SCMP_ACT_ALLOW",
            "args": null
        }, ...
    ]
}

Then, maybe it's permissions, NodeJS, npm? Nope, I'm currently running the process as root, have cleaned out node_modules, put a ton of logging in the Jenkinsfile to see what's happening. I'm using a Jenkins pipeline to run the build - these are the checkout, logging and build steps:

checkout scm
sh 'rm -rf ./node_modules'
sh 'npm install'
sh 'mkdir -p ./dist/my-web-app/assets'
sh 'chmod 766 ./dist/my-web-app/assets'
sh 'ls -al ./dist/my-web-app'
sh 'whoami'
sh 'pwd'
sh 'ls -al'
sh 'node --version'
sh 'npm --version'
sh 'ls -al /var/jenkins_home/workspace/My-Web-App/src/assets'
sh 'ng build --configuration=staging'

which results in the following log:

Building...
[Pipeline] sh
+ mkdir -p ./dist/my-web-app/assets
[Pipeline] sh
+ chmod 766 ./dist/my-web-app/assets
[Pipeline] sh
+ ls -al ./dist/my-web-app
total 5848
.....
drwxrw-rw- 2 root root    4096 Jun 23 17:44 assets
.....
[Pipeline] sh
+ whoami
root
[Pipeline] sh
+ pwd
/var/jenkins_home/workspace/My-Web-App
[Pipeline] sh
+ ls -al
total 720
.....
drwxr-xr-x   3 root root   4096 Jun 23 17:43 dist
-rw-r--r--   1 root root     95 Feb 28  2020 docker-compose.yml
-rw-r--r--   1 root root   3522 Jun 23 16:48 Jenkinsfile
.....
[Pipeline] sh
+ node --version
v14.17.1
[Pipeline] sh
+ npm --version
6.14.13
[Pipeline] sh
+ ls -al /var/jenkins_home/workspace/My-Web-App/src/assets
total 2080
drwxr-xr-x 2 root root    4096 Jun 23 18:39 .
drwxr-xr-x 6 root root    4096 Jun 23 18:39 ..
-rw-r--r-- 1 root root    3737 Feb 28  2020 building.svg
.....
[Pipeline] sh
+ ng build --configuration=staging
- Generating browser application bundles (phase: setup)...
Compiling @angular/core : es2015 as esm2015
.....
✔ Browser application bundle generation complete.
- Generating ES5 bundles for differential loading...
✔ Browser application bundle generation complete.
✔ ES5 bundle generation complete.
- Copying assets...
✖ Copying of assets failed.
Unable to copy assets: EPERM: operation not permitted, 
copyfile '/var/jenkins_home/workspace/My-Web-App/src/assets/building.svg' 
-> '/var/jenkins_home/workspace/My-Web-App/dist/my-web-app/assets/building.svg'

The build successfully creates the main-...js files etc in the dist folder, but just doesn't copy the assets. I can successfully create a file then copy it in the Jenkinsfile sh, but not during the Angular build:

[Pipeline] sh
+ touch ./dist/my-web-app/assets/me.txt
[Pipeline] sh
+ cp ./dist/my-web-app/assets/me.txt ./dist/my-web-app/assets/me-copy.txt
[Pipeline] sh
+ ls -al ./dist/my-web-app/assets
total 8
drwxrw-rw- 2 root root 4096 Jun 24 10:40 .
drwxr-xr-x 3 root root 4096 Jun 23 18:43 ..
-rw-r--r-- 1 root root    0 Jun 24 10:40 me-copy.txt
-rw-r--r-- 1 root root    0 Jun 24 10:40 me.txt

Maybe it's the Angular version? I've tried the same thing with Angular 9, 10, 11 and 12, with the same results, so it looks like it's not Angular causing the issue. The Angular folks suggest it's not an Angular issue: https://github.com/angular/angular-cli/issues/21198

angular.config (configurations)

"configurations": {


    "staging": {
        "fileReplacements": [
            {
                "replace": "src/environments/environment.ts",
                "with": "src/environments/environment.staging.ts"
            }
        ],
        "optimization": true,
        "outputHashing": "all",
        "sourceMap": false,
        "namedChunks": false,
        "extractLicenses": true,
        "vendorChunk": false,
        "buildOptimizer": true,
        "budgets": [
            {
                "type": "initial",
                "maximumWarning": "2.7mb",
                "maximumError": "15mb"
            },
            {
                "type": "anyComponentStyle",
                "maximumWarning": "8kb",
                "maximumError": "10kb"
            }
        ]
    },
    ...

Node, NPM? I've tried with Node 12 and 14. I've tried npm cache clean --force, npm cache verify and as shown above, also deleting the entire node_modules folder.

Docker? I've rebuilt the docker image and container multiple times using jenkins/jenkins:latest as the base

I've read and tried the plethora of StackOverflow, Github etc comments from folks with similar issues suggesting restarting, rebuilding, checking perms, upgrading etc but no luck, same issue.

Not sure where the issue is: Angular, Docker, Jenkins, config, perms, colour-of-socks-im-wearing... Any advice gratefully accepted.