The filename of my curl download target is unpredictable and globbing with an asterisk isn't possible. I can download the file using the following command, but only after I've determined its' name in advance:
curl -O -vvv -k -u user:password https://myURL/ws/myfile.zip
How can I tailor my curl command to succeed with an unpredictable target name?
There's no easy way to get a directory listing using HTTP. You can use curl to just print the HTML generated by the site. If there's an index with links to the files on that server, simply running
curl -s -u user:password https://myURL/ws/ | grep .zip
will print HTML-formatted links to the zip files available for download on that page.
Like the OP, I had a similar issue scripting the download of a binary- for docker-compose- from Github because the version number keeps iterating making the file name unpredictable.
This is how I solved it. Might not be the tidiest solution, but if you have a more elegant way, ping me a comment and I'll update the answer.
I merely used an auto-populating variable that takes the output of curl, prints the 1st line- which will be the most recent release- and thengrep for the release number prefaced by a "v". The result is saved to the the path /home/ubuntu as the arbitrary file name "docker-compose-latest"
curl -L "https://github.com/docker/compose/releases/download/$(curl https://github.com/docker/compose/releases | grep -m1 '<a href="/docker/compose/releases/download/' | grep -o 'v[0-9:].[0-9].[0-9]')/docker-compose-$(uname -s)-$(uname -m)" -o /home/ubuntu/docker-compose-latest
And we validate that we received the correct binary (I'm downloading to a Raspberry Pi which has an ARM processor on 64 bit Ubuntu 20.04 LTS:
file /home/ubuntu/docker-compose-latest
Produces the following feedback on the file:
/home/ubuntu/docker-compose-latest: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=QqyJMzYMWOofWehXt3pb/T7U4zg-t8Xqz_11RybNZ/ukJOlZCpzQuZzBcwSK3b/d6ecQ2m2VfqKb_EQRUZA, stripped
To validate this solution works, just execute the above commands remembering to change the path of the file command if not using Ubuntu.
Again, might not be the most elegant solution, but it's a solution for how one can download a target with curl that has an unpredictable filename.