Gitlab secret detection, how to test it works

If the secrets detector finds a secret, it doesn't fail the job (ie, it doesn't have a non-0 exit code). In the analyzer output, it will show how many leaks were found, but not what they were. The full details are written to a file called gl-secret-detection-report.json. You can either cat the file in the job so you can see the results in the Job output, or upload it as an artifact so it gets recognized as a sast report.

Here's the secrets detection job from one of my pipelines that both cat's the file and uploads it as a sast report artifact. Note: for my purposes, I wasn't able to directly use the template, so I run the analyzer manually:

Secrets Detector:
  stage: sast
  image:
    name: "registry.gitlab.com/gitlab-org/security-products/analyzers/secrets"
  needs: []
  only:
    - branches
  except:
    - main
  before_script:
    - apk add jq
  script:
    - /analyzer run
    - cat gl-secret-detection-report.json | jq '.'
  artifacts:
    reports:
      sast: gl-secret-detection-report.json

The gl-secret-detection-report.json file looks like this for a test repository I set up and added a GitLab Runner registration token to a file called TESTING:

{
  "version": "14.0.4",
  "vulnerabilities": [
    {
      "id": "138bf52be327e2fc3d1934e45c93a83436c267e45aa84f5b55f2db87085cb205",
      "category": "secret_detection",
      "name": "GitLab Runner Registration Token",
      "message": "GitLab Runner Registration Token detected; please remove and revoke it if this is a leak.",
      "description": "Historic GitLab Runner Registration Token secret has been found in commit 0a4623336ac54174647e151186c796cf7987702a.",
      "cve": "TESTING:5432b14f2bdaa01f041f6eeadc53fe68c96ef12231b168d86c71b95aca838f3c:gitlab_runner_registration_token",
      "severity": "Critical",
      "confidence": "Unknown",
      "raw_source_code_extract": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "scanner": {
        "id": "gitleaks",
        "name": "Gitleaks"
      },
      "location": {
        "file": "TESTING",
        "commit": {
          "author": "author",
          "date": "2022-09-12T17:30:33Z",
          "message": "a commit message",
          "sha": "0a4623336ac54174647e151186c796cf7987702a"
        },
        "start_line": 1
      },
      "identifiers": [
        {
          "type": "gitleaks_rule_id",
          "name": "Gitleaks rule ID gitlab_runner_registration_token",
          "value": "gitlab_runner_registration_token"
        }
      ]
    }
  ],
  "scan": {
    "analyzer": {
      "id": "secrets",
      "name": "secrets",
      "url": "https://gitlab.com/gitlab-org/security-products/analyzers/secrets",
      "vendor": {
        "name": "GitLab"
      },
      "version": "4.3.2"
    },
    "scanner": {
      "id": "gitleaks",
      "name": "Gitleaks",
      "url": "https://github.com/zricethezav/gitleaks",
      "vendor": {
        "name": "GitLab"
      },
      "version": "8.10.3"
    },
    "type": "secret_detection",
    "start_time": "2022-09-12T17:30:54",
    "end_time": "2022-09-12T17:30:55",
    "status": "success"
  }
}

This includes the type of secret found, what file it was in and what line(s), and information from the commit where the secret was added.

If you wanted to force the job to fail if any secrets were found, you can do that with jq (note: I install jq in the before_script of this job, it's not available in the image by default.):

Secrets Detector:
  stage: sast
  image:
    name: "registry.gitlab.com/gitlab-org/security-products/analyzers/secrets"
  needs: []
  only:
    - branches
  except:
    - main
  before_script:
    - apk add jq
  script:
    - /analyzer run
    - cat gl-secret-detection-report.json | jq '.'
    - if [[ $(cat gl-secret-detection-report.json | jq '.vulnerabilities | length > 0') ]]; then echo "secrets found" && exit 1; fi
  artifacts:
    reports:
      sast: gl-secret-detection-report.json