How do I know how much data a `docker pull` will fetch?

user_2396993 0
Viewed 233

Say I have a local build that's been pushed to the registry.

One update has been made that changes just a couple of the layers.

I need to fetch those updates, and Docker will know internally which layers it needs to pull, so it knows internally how much data will be pulled.

I can't see a way for me to find out before I do the pull, how much data will be pulled, i.e. how large the 2 modified layers are.

All the methods I have seen about inspecting layers locally and on the registry are not helpful, since the registry shows layer hashes for gzip'd layers, and the local docker shows hashes for un-gzip'd layers, so they are not comparable. The registry therefor has a list of hashes that match NONE of my local layer hashes.

I think I have a couple of options:

a) calculate the COMPRESSED layer hashes locally, to compare to the registry

b) use a third party server to download the layers from the registry and calculate the LOCAL hashes from the fetched layers, so I know their size.

docker-registry docker
2 Answers

if your images are stored in docker hub, you can hit the "https://hub.docker.com/v2/repositories/library/<image-name>/tags/?page_size=25" endpoint and fetch all the details of your image.

e.g if i do a curl -s https://hub.docker.com/v2/repositories/library/hello-world/tags i get a response like:

{
  "count": 9,
  "next": "https://hub.docker.com/v2/repositories/library/hello-world/tags/?page=2&page_size=2",
  "previous": null,
  "results": [
    {
      "creator": 1156886,
      "id": 164125618,
      "image_id": null,
      "images": [
        {
          "architecture": "amd64",
          "features": "",
          "variant": null,
          "digest": "sha256:71ca70650c09da31487c1dfb348bf780c8807b9617974180eaaed46381e82ab0",
          "os": "windows",
          "os_features": "",
          "os_version": "10.0.20348.169",
          "size": 117982763,
          "status": "active",
          "last_pulled": "2021-08-25T08:46:14.795809Z",
          "last_pushed": "2021-08-24T23:43:26.950422Z"
        }
      ],
      "last_updated": "2021-08-24T23:43:53.833792Z",
      "last_updater": 1156886,
      "last_updater_username": "doijanky",
      "name": "nanoserver-ltsc2022",
      "repository": 31760,
      "full_size": 117982763,
      "v2": true,
      "tag_status": "active",
      "tag_last_pulled": "2021-08-25T08:46:14.795809Z",
      "tag_last_pushed": "2021-08-24T23:43:53.833792Z"
    },
    {
      "creator": 2215,
      "id": 6273972,
      "image_id": null,
      "images": [
        {
          "architecture": "amd64",
          "features": "",
          "variant": null,
          "digest": "sha256:07ec39351767b7d0453681761b5a7e1bbc98b71243ad5db8481af4700f7cc06e",
          "os": "windows",
          "os_features": "",
          "os_version": "10.0.17763.2114",
          "size": 102744045,
          "status": "active",
          "last_pulled": "2021-08-25T08:46:14.915481Z",
          "last_pushed": "2021-08-24T21:42:07.39618Z"
        },
        {
          "architecture": "amd64",
          "features": "",
          "variant": null,
          "digest": "sha256:71ca70650c09da31487c1dfb348bf780c8807b9617974180eaaed46381e82ab0",
          "os": "windows",
          "os_features": "",
          "os_version": "10.0.20348.169",
          "size": 117982763,
          "status": "active",
          "last_pulled": "2021-08-25T08:46:14.795809Z",
          "last_pushed": "2021-08-24T23:43:26.950422Z"
        }
      ],
      "last_updated": "2021-08-24T23:43:47.337855Z",
      "last_updater": 1156886,
      "last_updater_username": "doijanky",
      "name": "nanoserver",
      "repository": 31760,
      "full_size": 117982763,
      "v2": true,
      "tag_status": "active",
      "tag_last_pulled": "2021-08-25T08:46:14.915481Z",
      "tag_last_pushed": "2021-08-24T23:43:47.337855Z"
    }
  ]
}

Now, you should be able to filter out any two images from the results array using some info (name/tag/hash/date) and subtract the sizes between the two. In your case, the one you have locally and the one you will be pulling (i.e the latest one). The size difference should give you an idea on how much data will be fetched if you pull the modified layer.

Also, check out the official documentation of the docker hub api: Here

user_104984180

You can try Skopeo. This tool allows you to inspect an image without pulling it first.

Inspecting a remote image showing its properties including its layers, without requiring you to pull the image to the host.

An example from Skopeo-inspect command

$ skopeo inspect docker://docker.io/fedora
{
    "Name": "docker.io/library/fedora",
    "Digest": "sha256:a97914edb6ba15deb5c5acf87bd6bd5b6b0408c96f48a5cbd450b5b04509bb7d",
    "RepoTags": [
        "20",
        "21",
        "22",
        "23",
        "24",
        "heisenbug",
        "latest",
        "rawhide"
    ],
    "Created": "2016-06-20T19:33:43.220526898Z",
    "DockerVersion": "1.10.3",
    "Labels": {},
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:7c91a140e7a1025c3bc3aace4c80c0d9933ac4ee24b8630a6b0b5d8b9ce6b9d4"
    ]
}
user_75028000
Related