List all the projects and all the groups

In bash for Gitlab API V4:

#!/bin/bash
GL_DOMAIN=""
GL_TOKEN=""
echo "" > gitlab_projects_urls.txt
for ((i=1; ; i+=1)); do
    contents=$(curl "$GL_DOMAIN/api/v4/projects?private_token=$GL_TOKEN&per_page=100&page=$i")
    if jq -e '. | length == 0' >/dev/null; then 
       break
    fi <<< "$contents"
    echo "$contents" | jq -r '.[].ssh_url_to_repo' >> gitlab_projects_urls.txt
done

Using the python-gitlab module we can get all groups and projects inside it. But if you have more than 20 groups or 20 projects inside of any group use pagination.

import gitlab
gl = gitlab.Gitlab('https://gitlab.com', private_token='Token')
groups = gl.groups.list()
for each in groups:
    group = gl.groups.get(each, lazy=True)
    project_lst=group.projects.list(as_list=False)  #pagination
    for item in project_lst:
    project_id = gl.projects.get(item.attributes['id'])
        ......................................
        ......................................

Here's a small shell script to get all GitLab projects:

echo -n "Please enter private token: "
read -s PRIVATE_TOKEN
number_of_pages=$(curl -s --head "https://gitlab.example.org/api/v4/projects?private_token=$PRIVATE_TOKEN" | grep -i x-total-pages | awk '{print $2}' | tr -d '\r\n')
for page in $(seq 1 $number_of_pages); do
    curl -s "https://gitlab.example.org/api/v4/projects?private_token=$PRIVATE_TOKEN&page=$page"
done

Replace gitlab.example.org with the hostname of your GitLab server.

Adjust it as needed; for example, this uses jq to get just the SSH repo URL for each project:

for page in $(seq 1 $number_of_pages); do
    curl -s "https://gitlab.example.org/api/v4/projects?private_token=$PRIVATE_TOKEN&page=$page" | jq -r '.[] | .ssh_url_to_repo'
done

Using python You can easily List all the groups and the Projects.Below is the code which you can try to list the project and groups:

import os
import gitlab

gl = gitlab.Gitlab('http://gitlab_hostname.com', 'your_private_token')
groups = gl.groups.list()
projects = gl.projects.list()
all_projects = gl.projects.list(all=True)
all_groups=gl.groups.list(all=True)
print("All groups are:",all_groups)
length=len(all_projects)
i=0
while i < length:
    project = gl.projects.get(all_projects[i].id)
    print(project)
    i=i+1

With python-gitlab you can do this with:

$ gitlab project list                                                                                                 [6:53:23]
id: 58
path: project-omega

id: 56
path: sigma

id: 54
path: theta

...

Use the graphql endpoint

It is very efficient to use the graphql endpoint of gitlab because you can get structured json result in a very concise and resource efficient way, here are two exemplary steps to achieve the task of "listing all projects and groups":

1. Use the graphql-explorer to find your query
2. Send requests to the graphql endpoint with your developed query (python example)

Use the graphql-explorer to find your query

1. Login to your gitlab instance (logging in is a required so that the queries also work on your private entities)
2. Go to the graphql explorer https://gitlab.com/-/graphql-explorer

After a bit of experimenting you would eventually come up with a query similar to this:

query {
    user(username:"YourUserName") {
    groups {
      nodes {
        id
        parent {
          id
        }
        name
        descendantGroups {
          nodes {
            id
            name
          }
        }
        projects {
          nodes {
            id
            name
          }
        }
      }
    }
  }
}

Send requests to the graphql endpoint with your developed query

Now you can e.g. via python post the above developed queries and work on the returned data:

import requests

query = """
query {
    user(username:"YourUserName") {
    groups {
      nodes {
        id
        parent {
          id
        }
        name
        descendantGroups {
          nodes {
            id
            name
          }
        }
        projects {
          nodes {
            id
            name
          }
        }
      }
    }
  }
}
"""
url = 'https://gitlab.com/api/graphql?private_token=your_private_token'
response = requests.post(url, json={'query': query})
data = response.json()
# work on data ...

Happy coding