Your best bet for generating the range of valid dates/times will probably come from combining two ideas:
- use epoch seconds for comparisons and math
- use
awk (or comparable program) to replace the time-consuming bash/while loop
One epoch(secs) / awk idea:
strdate='2022-01-01T00:00'
enddate='2022-01-31T23:30'
strdate_s=$(date -d "${strdate}" +%s)
enddate_s=$(date -d "${enddate}" +%s)
inc_m=30
((inc_s = inc_m * 60))
awk -v ss="${strdate_s}" -v es="${enddate_s}" -v inc="${inc_s}" '
BEGIN { while ( ss <= es ) {
print strftime("%d%H%M", ss)
ss+=inc
}
}
' > date_correct
NOTE: as Fravadona's mentioned in the comments, strftime() requires GNU awk (aka gawk)
To show the performance improvement of using awk instead of the bash/while loop we'll modify OP's current code to use the epoch(secs) approach:
strdate='2022-01-01T00:00'
enddate='2022-01-31T23:30'
strdate_s=$(date -d "${strdate}" +%s)
enddate_s=$(date -d "${enddate}" +%s)
inc_m=30
((inc_s = inc_m * 60))
while [[ "${strdate_s}" -le "${enddate_s}" ]] ; do
date -d "@${strdate_s}" +%d%H%M >> date_correct2
((strdate_s+=inc_s))
done
A diff of the outputs show both sets of code generate the same output:
$ diff date_correct date_correct2
<<<=== no output
Results of running both processes under time:
# awk
real 0m0.042s
user 0m0.015s
sys 0m0.015s
# bash/while
real 0m46.412s
user 0m6.727s
sys 0m27.314s
So awk is about 1100x times faster than a comparable bash/while loop.
If the sole purpose of this date/time-generating code is simply to find the missing dates/times in the date_metar file then OP may want to consider using a single awk script to eliminate the need for the date_correct file and still determine what dates/times are missing from date_metar ... but that's for another Q&A ...
Looking a bit more into the performance issues of the bash/while loop ...
Replacing the date call with a comparable printf -v call:
while [[ "${strdate_s}" -le "${enddate_s}" ]] ; do
printf -v new_date '%(%d%H%M)T' "${strdate_s}"
echo "${new_date}" >> date_correct2
((strdate_s+=inc_s))
done
We see overall time is reduced from 46+ secs to 10+ secs:
real 0m10.127s
user 0m0.141s
sys 0m0.312s
We should be able to get a further improvement by moving the >> date_correct2 to after the done, thus replacing 1400+ file open/close operations (date ... >> date_correct2) with a single file open/close operation (done > date_correct2)
while [[ "${strdate_s}" -le "${enddate_s}" ]] ; do
printf -v new_date '%(%d%H%M)T' "${strdate_s}"
echo "${new_date}"
((strdate_s+=inc_s))
done > date_correct2
This speeds up the process by ~50x times (10+ secs down to 0.2 secs):
real 0m0.198s
user 0m0.141s
sys 0m0.000s
Thus reducing the bash/while loop overhead (compared to awk) from 1100x to 5x.