How to Bulk Download Azure Artifacts Packages?

user_3501875 0
Viewed 1306

My organization is moving our software packages (NuGet, NPM) to Azure Artifacts. We have approximately 50,000 NuGet packages (2GB) that we've pushed to Azure Artifacts.

Is there a way to to bulk-pull my packages from Azure Artifacts either to my machine or to Azure blob storage in case we are not satisfied with the product? I'm mainly concerned about pulling our NuGet packages because we have thousands of packages published.

I'm aware of Download all packages from private nuget feed but I'd like to know if there's a way to retrieve a zip file of all of our packages, or some other method to download all of our packages in bulk.

Thanks!

azure-artifacts azure-devops azure
2 Answers

We don't have a bulk download option, but it would be pretty easy to script something yourself.

First, you'll want to get a PAT. Make sure the PAT has the Packaging (read) scope. You will use the PAT as a password for authentication. The username you use doesn't matter.

Then, you'll need to get the list of all packages in the feed. You can use our Get Packages API for that.

Then, for NuGet packages, the protocol is as follows:

  1. Download the feed URL (from the Connect to Feed dialog, will end with .../v3/index.json)
  2. Read the returned JSON document, get the @id attribute from the object with @type = PackageBaseAddress/3.0.0. This is the PackageBaseAddress. Note that the actual value is considered an implementation detail. Always retrieve the address from index.json.
  3. If you do not already have the list of versions, download {PackageBaseAddress}/{PackageName (lowercased)}/index.json which will have the list as a JSON array (I suggest getting the versions this way since they will already be normalized)
  4. Download the packages from {PackageBaseAddress}/{PackageName (lowercase)}/{PackageVersion (normalized)}/{PackageName (lowercase)}.{PackageVersion (normalized)}.nupkg
    • e.g. {PackageBaseAddress}/restsharp/106.4.1/restsharp.106.4.1.nupkg
    • Be sure to follow redirects, as the service will actually send you to download from our backend storage

For npm packages:

  1. Get the Feed URL from the Connect to Feed dialog (it will end with .../npm/registry)
  2. Download {Feed URL}/{PackageName} (for scoped packages include the scope, e.g. {Feed URL}/@{Scope}/{PackageName})
  3. Download URLs for each version are listed in the JSON response as versions.*.dist.tarball
  4. Again, be sure to handle redirects when downloading the packages.
user_66589310

Simplest and easiest way to bulk download azure artifacts are using ui or website. Login and go to azure artifacts, select you want to download or select all. Initially when you click download you will see that only one the first one downloaded. The reason is browser automatically blocked popups. So you will see popup closed option on url bar , just allow popup. Now once you click download it will start downloading all. Suggestion: change download location to a different path.

user_154306670
Related