Conan local cache when using with Jenkins

user_13715414 0
Viewed 1698

Our Jenkins pipelines will have different concurrent jobs running conan install and conan export-pkg which update the local cache.

We currently have one single local cache for the jenkins user in /home/jenkins/conan. Multiple users login to Jenkins and thus what one user does will affect the others.

What is the preferred way to use Jenkins and Conan when it comes to local cache? Should every Jenkins job get its own local cache by setting CONAN_USER_HOME=$WORKSAPCE/conan or something like that?

conan jenkins
1 Answers

The Conan cache is not designed for concurrency. If one job is creating and building a package and at the same time another job tries to use that package, there can be race conditions, and jobs might fail unexpectedly and with undefined behavior. So concurrent usage is not allowed, and this rules out to share the cache for general purpose jobs, unless your usage patterns are extremely controlled.

For most cases then, it is preferred to setup a clean cache per job, via the CONAN_USER_HOME environment variable.

There are several things that can be done to have faster builds:

  • Having a server, like an Artifactory close to the CI machines, with the precompiled binaries for the dependencies.
  • There is a new experimental download cache that is concurrent and can be shared among multiple CI jobs. It can have great benefits specially if the networks is slow.
user_32153830
Related