How to specify depth in checkout(scm) in scripted pipeline

user_3270950 0
Viewed 6032

our project repo is very big (2.5GB). Hence upon checkout(scm) step in the scripted pipeline, the code takes longer time to clone from GIT. and We are facing the below errors, due to GIT trying to fetch the entire history.

I have tried so far with checkout(scm), I read in https://jenkins.io/doc/pipeline/steps/workflow-scm-step/ that there is an option called depth, by which we can download only the recent commits.

But i don't know the syntax for it. 

node(nodeName) {
    try {
    deleteDir()
    checkout(scm)
        ....
        ....
    }
    catch(Exception ex) {
        throw ex
    }    
}

If the cloning times reduces, it would be a lot beneficial. Upon executing the line checkout(scm),

We are sometimes getting the below error.

using credential Servicejenkins_build
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository 
hudson.plugins.git.GitException: Command "C:\Git\cmd\git fetch --no-tags --progress https://gitlab.com/../../supportforpc.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: remote: Enumerating objects: 1           
remote: Enumerating objects: 24671, done.        
remote: Counting objects:   0% (1/24671)           
remote: Counting objects:   1% (247/24671)           
remote: Counting objects:   2% (494/24671)           
remote: Counting objects:   3% (741/24671)           
remote: Counting objects:   4% (987/24671)           
remote: Counting objects:   5% (1234/24671)           
remote: Counting objects:   6% (1481/24671)  
.....
....
Counting objects:   100% (24671/24671) 
remote: Compressing objects:   0% (1/10279)           
remote: Compressing objects:   1% (103/10279)           
remote: Compressing objects:   2% (206/10279)           
remote: Compressing objects:   3% (309/10279)           
remote: Compressing objects:   4% (412/10279)           
remote: Compressing objects:   5% (514/10279)           
remote: Compressing objects:   6% (617/10279)           
remote: Compressing objects:   7% (720/10279)           
remote: Compressing objects:   8% (823/10279)           
remote: Compressing objects:   9% (926/10279)           
remote: Compressing objects:  10% (1028/10279) 
....
....
remote: Compressing objects:  100% (10279/10279) 
Receiving objects:   0% (1/24671)   
Receiving objects:   1% (247/24671)   
Receiving objects:   2% (494/24671)   
Receiving objects:   3% (741/24671)   
Receiving objects:   4% (987/24671)   
Receiving objects:   5% (1234/24671)   
Receiving objects:   6% (1481/24671)
....
....
Receiving objects:   43%

    fatal: index-pack failed
    error: RPC failed; curl 56 SSL read: 
    error:00000000:lib(0):func(0):reason(0), errno 10054

Hence I thought, using depth 1 in checkout(scm) might solve the issue. But i dont know the syntax in scripted pipeline.

jenkins-groovy jenkins-pipeline jenkins
2 Answers

You can enable a shallow clone (no history and only the most recent commit is fetched) in the scm object before the checkout:

node(nodeName) {
    try {
    deleteDir()
    scm.extensions << [$class: 'CloneOption', shallow: true]
    checkout(scm)
        ....
        ....
    }
    catch(Exception ex) {
        throw ex
    }    
}

However, I would advise you to set up and maintain a reference repository, which is faster in an order of magnitude:

  1. In a terminal window clone your repository into a mirror. This repository will only contain git objects:

    $ git clone --mirror https://gitlab.com/../../supportforpc.git
Cloning into bare repository 'supportforpc.git'...
remote: Enumerating objects: 6578, done.
remote: Counting objects: 100% (6578/6578), done.
remote: Compressing objects: 100% (1561/1561), done.
remote: Total 739260 (delta 5791), reused 5046 (delta 5013), pack-reused 732682
Receiving objects: 100% (739260/739260), 3.49 GiB | 3.78 MiB/s, done.
Resolving deltas: 100% (562236/562236), done.

  2. Create a new job in Jenkins to update the mirror repository periodically. Use only the fetch command when updating the mirror:

    sh "git fetch --all --prune"

  3. Tell the scm object to make use of the mirror repository as reference. The remote repository is queried for new commits after the reference is read, so you don't have to worry about always keeping the mirror up-to-date:

    node(nodeName) {
    try {
    deleteDir()
    scm.extensions << [$class: 'CloneOption', reference: "<your-server>:git/<where-you-put-the-mirror-repo>"]
    checkout(scm)
        ....
        ....
    }
    catch(Exception ex) {
        throw ex
    }    
}

After that setup you'll see that your whole repository is cloned in a matter of seconds, plus you get to keep all the repository's history. You can test the clone for yourself if you manually clone from the mirror repository you created in step 1:

$ git clone <mirror-repository-directory> <some-dir>`
Cloning into '<some-dir>'...
done.
Checking out files: 100% (15055/15055), done.

A note regarding the modification of the default scm object: if you prefer not to change it you can create a new object for the checkout as shown in this answer:

checkout([
    $class: 'GitSCM',
    branches: scm.branches,
    doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
    extensions: scm.extensions + [$class: 'CloneOption', reference: "<your-server>:git/<where-you-put-the-mirror-repo>"],
    userRemoteConfigs: scm.userRemoteConfigs
])
user_64269760 
checkout changelog: false, poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/branchname'], [$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: true]], userRemoteConfigs: [[credentialsId: 'credentialId', url: 'git@repourl']]]

credentialId can be found by hitting http://yourjenkinsinstall/credentials

user_119604980
Related