How to run multiple commands parallelly within a ssh session on a storage appliance

Viewed 42

I was helping one of the SO member in my acquaintance in order to enable him to execute the script which is working for him, however as this script will run 80k command over all aggregate based on the volume list from the file called volumes.csv which is huge.

As this Storage Appliance hence this doesn't support all ssh attributes just for clarification, however this looks like as below when you get ssh connection into the Appliance and then we used to run commands as follows.

$ ssh dbcl101
dbcl101::> row 0;set -unit MB
dbcl101::> vol show fxn-backupvol -fields performance-tier-inactive-user-data
vserver volume                             performance-tier-inactive-user-data
------- ---------------------------------- -----------------------------------
stv3008 fxn-backupvol -

Example: for an example below are the 10 volumes that starts with ^fxn thus these Volumes will be a part of above command which in turn will run one by one 10 times on the dbcl101 and in reality there are odd 40k volumes as per the data which will take hours, but i did not get any smart way myself as of now to run these commands parallelly on the command shell of appliance.

fxn-backupvol1
fxn-backupvol2
fxn-backupvol3
fxn-backupvol4
fxn-backupvol5
fxn-backupvol6
fxn-backupvol7
fxn-backupvol8
fxn-backupvol9
fxn-backupvol0

Below is the running code:

#!/bin/bash
#####################################################################################################
# Create log file in a incremental fashion
# The format string for the printf, %s-%02d.txt, means "a string followed by a literal dash
# followed by a zero-filled two-digit integer
#######################################################################################################
PATH="/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin"
export $PATH
today=$( date +%Y%m%d )
number=0
datafile="/home/archive_dashboard/storage_footprint-$today"
while [ -e "$datafile" ]; do
    printf -v datafile '%s-%02d' "/home/archive_dashboard/storage_footprint-$today" "$(( ++number ))"
done
printf 'Will use "%s" as filename\n' "$datafile"
touch "$datafile"
#------------- logfile creation ends here ---------
projects=$(awk -F ',' ' BEGIN{getline; NR--} {gsub(/"/,""); printf $2":"$NF"\n"}' volumes.csv );
for proj in $(echo $projects);
do
vols=$(echo $proj | awk -F: '{print $2}');
ldap_proj=$(echo $proj | awk -F: '{print $1}');
     [[ $vols =~ fxn1 ]] && ssh_server="dbcl101";
     [[ $vols =~ fxn2 ]] && ssh_server="dbcl201";
     [[ $vols =~ fxn3 ]] && ssh_server="dbcl301";
     [[ $vols =~ fxn4 ]] && ssh_server="dbcl401";
     [[ $vols =~ fxn1 ]] && ssh_server="dbcl101";
     [[ $vols =~ fxn2 ]] && ssh_server="dbcl201";
     [[ $vols =~ fxn3 ]] && ssh_server="dbcl301";
     [[ $vols =~ fxn4 ]] && ssh_server="dbcl401";
     [[ $vols =~ fxn5 ]] && ssh_server="dbcl501";
     [[ $vols =~ fxn6 ]] && ssh_server="dbcl601";
     [[ $vols =~ fxn7 ]] && ssh_server="dbcl701";
     [[ $vols =~ fxn8 ]] && ssh_server="dbcl804";
     [[ $vols =~ fxn017 ]] && ssh_server="dbcl031";
     [[ $vols =~ fxn015 ]] && ssh_server="dbcl032";
     [[ $vols =~ fxn016 ]] && ssh_server="dbcl034";
     [[ $vols =~ fxn014 ]] && ssh_server="dbcl028";
       volstyle=$(ssh "$ssh_server" "row 0; vol show $vols -fields volume-style-extended" |awk '/fxn/{print $3}')
       if [[ "$volstyle" == "flexvol" && ! -z "$ssh_server" ]];then
         echo -n "$ldap_proj $vols "
         ssh $ssh_server "row 0;set -unit MB; \
         vol show $vols -fields performance-tier-inactive-user-data; \
         vol show-footprint $vols -fields volume-blocks-footprint-bin0,volume-blocks-footprint-bin1,volume-blocks-footprint" \
         |awk '/fxn/{print $3, $4, $5}'

       elif [[ "$volstyle" == "flexgroup" && ! -z "$ssh_server" ]];then
         echo -n "$ldap_proj $vols "
         flexgroupvol=${vols}'__*'
         ssh $ssh_server "row 0;set -unit MB; \
         vol show $vols -fields performance-tier-inactive-user-data; \
         vol show-footprint $flexgroupvol -fields volume-blocks-footprint-bin0,volume-blocks-footprint-bin1,volume-blocks-footprint" \
         |awk '/^fxn/{ s1+=$3; s2+=$4; s3+=$5; if (FNR == 8) { print $3 } } END { print s1, s2, s3 }'
      fi
done | tee -a  "$datafile.log1"

sed -i -e 's/\r//g; s/-/0/g; s/MB//g'  "$datafile.log1"
sed '/^[[:space:]]*$/d' "$datafile.log1"| paste -d " " - - > "$datafile.log2"
awk '{$1=$1}1' OFS="," "$datafile.log2" > "${datafile}"
sed -i '1s/^/Project_Name,Volume_Name,InactiveTier,PhysicalBlocks,SSDtier,GRIDtier\n/' "${datafile}"
rm -rf "$datafile.log2" "$datafile.log1"
exit;

Please suggest any better way of improvising this which will help myself and other folks around the stack community, much power to you guys in advance.

0 Answers
Related