Multiple simultaneous downloads using Wget?

use

aria2c -x 10 -i websites.txt >/dev/null 2>/dev/null &

in websites.txt put 1 url per line, example:

https://www.example.com/1.mp4
https://www.example.com/2.mp4
https://www.example.com/3.mp4
https://www.example.com/4.mp4
https://www.example.com/5.mp4

use xargs to make wget working in multiple file in parallel

#!/bin/bash

mywget()
{
    wget "$1"
}

export -f mywget

# run wget in parallel using 8 thread/connection
xargs -P 8 -n 1 -I {} bash -c "mywget '{}'" < list_urls.txt

Aria2 options, The right way working with file smaller than 20mb

aria2c -k 2M -x 10 -s 10 [url]

-k 2M split file into 2mb chunk

-k or --min-split-size has default value of 20mb, if you not set this option and file under 20mb it will only run in single connection no matter what value of -x or -s

You can use xargs

-P is the number of processes, for example, if set -P 4, four links will be downloaded at the same time, if set to -P 0, xargs will launch as many processes as possible and all of the links will be downloaded.

cat links.txt | xargs -P 4 -I{} wget {}

I'm using gnu parallel

cat listoflinks.txt | parallel --bar -j ${MAX_PARALLEL:-$(nproc)} wget -nv {}

1. cat will pipe a list of line separated URLs to parallel
2. --bar flag will show parallel execution progress bar
3. MAX_PARALLEL env var is for maximum no of parallel download, use it carefully, default here is current no of CPUs

tip: use --dry-run to see what will happen if you execute command.
cat listoflinks.txt | parallel --dry-run --bar -j ${MAX_PARALLEL} wget -nv {}

Consider using Regular Expressions or FTP Globbing. By that you could start wget multiple times with different groups of filename starting characters depending on their frequency of occurrence.

This is for example how I sync a folder between two NAS:

wget --recursive --level 0 --no-host-directories --cut-dirs=2 --no-verbose --timestamping --backups=0 --bind-address=10.0.0.10 --user=<ftp_user> --password=<ftp_password> "ftp://10.0.0.100/foo/bar/[0-9a-hA-H]*" --directory-prefix=/volume1/foo &
wget --recursive --level 0 --no-host-directories --cut-dirs=2 --no-verbose --timestamping --backups=0 --bind-address=10.0.0.11 --user=<ftp_user> --password=<ftp_password> "ftp://10.0.0.100/foo/bar/[!0-9a-hA-H]*" --directory-prefix=/volume1/foo &

The first wget syncs all files/folders starting with 0, 1, 2... F, G, H and the second thread syncs everything else.

This was the easiest way to sync between a NAS with one 10G ethernet port (10.0.0.100) and a NAS with two 1G ethernet ports (10.0.0.10 and 10.0.0.11). I bound the two wget threads through --bind-address to the different ethernet ports and called them parallel by putting & at the end of each line. By that I was able to copy huge files with 2x 100 MB/s = 200 MB/s in total.

Call Wget for each link and set it to run in background.

I tried this Python code

with open('links.txt', 'r')as f1:      # Opens links.txt file with read mode
  list_1 = f1.read().splitlines()      # Get every line in links.txt
for i in list_1:                       # Iteration over each link
  !wget "$i" -bq                       # Call wget with background mode

Parameters :

      b - Run in Background
      q - Quiet mode (No Output)