How to extract the list of all repositories in Stash or Bitbucket?

1. Once logged in: on the top right, click on your profile pic and then 'View profile'

2. Take note of your user (in the example below 'YourEmail@domain.com', but keep in mind it's case sensitive)

3. Click on profile pic > Manage account > Personal access token > Create a token (choosing 'Read' access type is enough for this functionality)

For all repos in all projects:

4. Open a CLI and use the command below (remember to fill in your server domain!):

curl -u "YourEmail@domain.com" -X GET https://<my_server_domain>/rest/api/1.0/projects/?limit=1000

5. It will ask you for your personal access token, you comply and you get a JSON file with all repos requested

For all repos in a given project:

4. Pick the project you want to get repos from. In my case, the project URL is: <your_server_domain>/projects/TECH/ and therefore my {projectKey} is 'TECH', which you'll need for the command below.

5. Open a CLI and use this command (remember to fill in your server domain and projectKey!):

curl -u "YourEmail@domain.com" -X GET https://<my_server_domain>/rest/api/1.0/projects/{projectKey}/repos?limit=50

Final touches

6. (optional) If you want just the titles of the repos requested and you have jq installed (for Windows, downloading the exe and adding it to PATH should be enough, but you need to restart your CLI for that new addition to be detected), you can use the command below:

curl -u $BBUSER -X GET <my_server_domain>/rest/api/1.0/projects/TECH/repos?limit=50 | jq '.values|.[]|.name'

(tested with Data Center/Atlassian Bitbucket v7.9.0 and powershell CLI)

I ended up having to do this myself with an on-prem install of Bitbucket which didn't seem to have the REST APIs discussed above accessible, so I came up with a short script to scrape it out of the web page. This workaround has the advantage that there's nothing you need to install, and you don't need to worry about dependencies, certs or logins other than just logging into your Bitbucket server. You can also set this up as a bookmark if you urlencode the script and prefix it with javascript:.

To use this:

1. Open your bitbucket server project page, where you should see a list of repos.
2. Open your browser's devtools console. This is usually F12 or ctrl-shift-i.
3. Paste the following into the command prompt there.

JSON.stringify(Array.from(document.querySelectorAll('[data-repository-id]')).map(aTag => {
  const href = aTag.getAttribute('href');
  let projName = href.match(/\/projects\/(.+)\/repos/)[1].toLowerCase();
  let repoName = href.match(/\/repos\/(.+)\/browse/)[1];
  repoName = repoName.replace(' ', '-');
  const templ = `https://${location.host}/scm/${projName}/${repoName}.git`;
  return {
    href,
    name: aTag.innerText,
    clone: templ
  }
}));

The result is a JSON string containing an array with the repo's URL, name, and clone URL.

[{
  "href": "/projects/FOO/repos/some-repo-here/browse",
  "name": "some-repo-here",
  "clone": "https://mybitbucket.company.com/scm/foo/some-repo-here.git"
}]

This ruby script isn't the greatest code, which makes sense, because I'm not the greatest coder. But it is clear, tested, and it works.

The script filters the output of a Bitbucket API call to create a complete report of all repos on a Bitbucket server. Report is arranged by project, and includes totals and subtotals, a link to each repo, and whether the repos are public or personal. I could have simplified it for general use, but it's pretty useful as it is.

There are no command line arguments. Just run it.

#!/usr/bin/ruby
#
# @author Bill Cernansky
#
# List and count all repos on a Bitbucket server, arranged by project, to STDOUT.
#
require 'json'

bbserver   = 'http(s)://server.domain.com'
bbuser     = 'username'
bbpassword = 'password'
bbmaxrepos = 2000       # Increase if you have more than 2000 repos

reposRaw = JSON.parse(`curl -s -u '#{bbuser}':'#{bbpassword}' -X GET #{bbserver}/rest/api/1.0/repos?limit=#{bbmaxrepos}`)

projects = {}

repoCount = reposRaw['values'].count

reposRaw['values'].each do |r|
    projID = r['project']['key']
    if projects[projID].nil?
        projects[projID] = {}
        projects[projID]['name'] = r['project']['name']
        projects[projID]['repos'] = {}
    end
    repoName = r['name']
    projects[projID]['repos'][repoName] = r['links']['clone'][0]['href']
end

privateProjCount = projects.keys.grep(/^\~/).count
publicProjCount = projects.keys.count - privateProjCount

reportText = ''
privateRepoCount = 0
projects.keys.sort.each do |p|
    # Personal project slugs always start with tilde
    isPrivate = p[0] == '~'
    projRepoCount = projects[p]['repos'].keys.count
    privateRepoCount += projRepoCount if isPrivate
    reportText += "\nProject: #{p} : #{projects[p]['name']}\n  #{projRepoCount} #{isPrivate ? 'PERSONAL' : 'Public'} repositories\n"

    projects[p]['repos'].keys.each do |r|
        reportText += sprintf("    %-30s : %s\n", r, projects[p]['repos'][r])
    end
end

puts "BITBUCKET REPO REPORT\n\n"
puts sprintf("  Total Projects: %5d     Public: %5d    Personal: %5d", projects.keys.count, publicProjCount, privateProjCount)
puts sprintf("  Total Repos:    %5d     Public: %5d    Personal: %5d", repoCount, repoCount - privateRepoCount, privateRepoCount)
puts reportText

The way I solved this issue, was get the html page and give it a ridiculous limit like this. thats in python :

 cmd = "curl -s  -k --user " +  username +  " https://URL/projects/<KEY_PROJECT_NAME>/?limit\=10000"

then I parsed it with BeautifulSoup

make_list = str((subprocess.check_output(cmd, shell=True)).rstrip().decode("utf-8"))
  html = make_list 
  parsed_html = BeautifulSoup(html,'html.parser')
list1 = [] 
  for a in parsed_html.find_all("a", href=re.compile("/<projects>/<KEY_PROJECT_NAME>/repos/")):
    list1.append(a.string)
print(list1)

to use this make sure you change and , this should be the bitbucket project you are targeting. All , I am doing is parsing an html file.

Here's how I pulled the list of repos from Bitbucket Cloud.

Setup OAauth Consumer

Go to your workspace settings and setup an OAuth consumer, you should be able to go here directly using this link: https://bitbucket.org/{your_workspace}/workspace/settings/api

The only setting that matters is the callback URL which can be anything but I chose http://localhost

Once setup, this will display a key and secret pair for your OAuth consumer, I will refer to these as {oauth_key} and {oauth_secret} below

Authenticate with the API

Go to https://bitbucket.org/site/oauth2/authorize?client_id={oauth_key}&response_type=code ensuring you replace {oauth_key}

This will redirect you to something like http://localhost/?code=xxxxxxxxxxxxxxxxxx, make a note of that code, I'll refer to that as {oauth_code} below

In your terminal go to curl -X POST -u "{oauth_key}:{oauth_secret}" https://bitbucket.org/site/oauth2/access_token -d grant_type=authorization_code -d code={oauth_code} replacing the placeholders.

This should return json including the access_token, I’ll refer to that access token as {oauth_token}

Get the list of repos

You can now run the following to get the list of repos. Bear in mind that your {oauth_token} lasts 2hrs by default.

curl --request GET \
  --url 'https://api.bitbucket.org/2.0/repositories/pageant?page=1' \
  --header 'Authorization: Bearer {oauth_token}' \
  --header 'Accept: application/json'

This response is paginated so you'll need to page through the responses, 10 repositories at a time.