Batch file to read folders in a share and append each folder name to the end of a path

user_1714301 0
Viewed 186

I'm trying to parallelise a script im using to backup some files using TSM.

Basically I run the following commands:

net use y: \\Server\share

dsmc incr y: -optfile="c:\path\folder\optionfile.opt" > c:\logs\logfile

.. and TSM backus up the entire directory tree in serial.

But I want to do this to create some seperate scripts to use more than one client to backup subsets of the directory tree (to speed things up)

FOR each subdirectory in \\Server\share

IF not processed 1/3 of the directories in \\server\share

DO



net use y: \\Server\share\**<append subdirectory here>**

Backup y

This is what I've got so far:

FOR /D %%a IN (W:) DO ( 

    net use y: \\server\share\%%a
    run command to backup y:

(y:\ should be now \server\share\first_dir_name)

    net use y: /delete

    timeout /t 10
)

I still need the if logic to count the number of folders in the share.

The first step was to find how many top level sub directories are in the share:

I use dir /a:d /s /b "Folder Path" | find /c ":\" but the command never completes.

Hope that makes sense!

Thanks.

batch-file
1 Answers

You could do something like this using semaphore files. In this example I have it set up for 8 separate tasks. I recommend that you copy this code and test to see how it works and then change the number of scripts you want to run simultaneously or figure out the optimum number to run simultaneously and change the code to keep that number of scripts running until done. You didn't say how many separate scripts you will have.

SeparateProcess.bat

@echo off
setlocal EnableDelayedExpansion

set "Script[1]=Script1"
set "Script[2]=Script2"
set "Script[3]=Script3"
set "Script[4]=Script4"
set "Script[5]=Script5"
set "Script[6]=Script6"
set "Script[7]=Script7"
set "Script[8]=Script8"
set /a nScripts=8

rem For test only. Change values (seconds) below if desired
set /A testDelay[1]=10
set /A testDelay[2]=4
set /A testDelay[3]=8
set /A testDelay[4]=14
set /A testDelay[5]=17
set /A testDelay[6]=6
set /A testDelay[7]=9
set /A testDelay[8]=7

rem time to wait between checking status of all tasks
set /A timeout=5
set /A tasksProcessing=0

rem Set all Status to 0 and delete semaphore files.
for /L %%L in (1,1,%nScripts%) do set /A Status[%%L]=0 & del /f /q  "SemaphoreFile!Script[%%L].txt!" 2>nul

echo This program will simulate starting separate processes
echo Open a Windows Explorer to %CD% and observe that semaphore files appear.
echo Observe that the files disappear when the processes complete after varying  delays.
pause

echo Starting %nScripts% processes
for /L %%L in (1,1,%nScripts%) do (
   echo( >"SemaphoreFile!Script[%%L]!.txt"
   start /MIN cmd.exe /C !Script[%%L]!.bat "SemaphoreFile!Script[%%L]!.txt" !testDelay[%%L]!
   set /A Status[%%L]=1
   set /A tasksProcessing+=1
   )

:KeepChecking
for /L %%A in (1,1,%nScripts%) do (
   echo Waiting %timeout% seconds for !tasksProcessing! processes to complete
   timeout %timeout% >nul
   for /L %%L in (1,1,%nScripts%) do (
      if !Status[%%L]!==1 if not exist "SemaphoreFile!Script[%%L]!.txt" (
         set /A Status[%%L]=0
         set /A tasksProcessing-=1
         echo !Script[%%L]! finished.
         )
      )
   if !tasksProcessing! GTR 0 goto :KeepChecking
   goto :Done
   )

:done
echo All processes completed.
pause
endlocal

Then have 8 scripts that are identical (for this test) where n is 1-8.

Scriptn.bat (code below)

@echo off
REM 1=Filespec of semaphor file
REM 2=Delay in seconds. This is for test only.
echo %~nx1 is doing something
timeout %~2
del "%~1"
got :eof

Execute SeparateProcess.bat to see how it works. When you are satisfied with the results replace the code in Script1-8 with whatever you need them to do. Just make sure that the last thing they do is delete the semaphore file to indicate that they are done.

user_20933520
Related