How to automatically clean-up workspace folders "...@script" and "...@libs" on master for Pipeline builds?

user_93039 0
Viewed 1316

When running Jenkins pipeline, some repositories are checked out on the master into the workspace folder, namely the project containing the Jenkinsfile, and also the repository for the shared library. They are stored in folders named after the project, but with suffix @script resp. @libs.

Once this is done, the Jenkinsfile gets interpreted and does all its work on agents (including cloning more repos etc.).

While it is easy to clean-up on the agent at the end of the build, this does not remove the files from the master. This way, we have assembled hundreds of GB of data over time.

This clean-up can't probably be done as part of the Jenkins build (configured in Jenkinsfile), because deleteDir() can only delete from the workspace on the agent. Maybe it would work by using java.util.File methods in script block (the Groovy code is executed on master), but still the Jenkinsfile and pipeline library might be locked as long as the build is running.

So I wonder if there is any other way how to do the master clean-up. Any plugin or script? Any other idea?

jenkins-pipeline jenkins
1 Answers

I must be missing something, but ran into a problem with this as well. Spent a morning searching but couldnt figure it out.

My use case is that I have shared library with a complete pipeline configurable through parameters. This pipeline is shared between quite a few buildjobs. (over 50). The pipeline makes use of a docker agent. And by setting it up like this, I get a pretty clean setup in jenkins / scripts / etc. Only thing is it leaves me with a cached copy of the docker image for each buildjob in the @libs folder on the master. This is quite the collection of Gigabytes sitting on the master doing nothing (well aside from speeding my next buildjob a few milliseconds).

In the end I created a separate buildjob that just deletes all these @libs folders on the master.

Its just a simple shell script job

#!/bin/bash

# This needs to be run occasionally. Can be run after every build.
# When using jenkins shared libraries with a pipeline with docker agent, 
# each job gets a local cached copy of the docker image. 
# which is about 1.4. Gb per job. 
# Which is a bit steep if you have dozens of jobs out there.

echo "Removing all temp docker images"
rm -fr ${WORKSPACE}/../*@libs/
ls -al ${WORKSPACE}/../

I have set this up to run at night. Doesnt seem to affect my builds, but im not sure you can use it also to get rid of the @tmp folders, as maybe jenkins might need that.

user_33824200
Related