Add file name to txt file if it is not zero bytes

user_8390764 0
Viewed 80

I would like to add files that meet a set of conditions to a txt file for easy transfer later, I do this with:

ls -1 > AllFilesPresent.txt

value=$(<AllFilesPresent.txt)

rm AllFilesPresent.txt

for val in $value; do
  case $val in
    (Result*.RData) echo "$val" >> CompletedJobs.txt ;;
  esac
done

I've run into the situation where some of the files are corrupted and show up as zero byte files, which i can find manually with:

find . -size 0 -maxdepth 1

How do I include adjust my loop to reject files that are zero bytes?

bash
2 Answers

The code

ls -1 > AllFilesPresent.txt
value=$(<AllFilesPresent.txt)
rm AllFilesPresent.txt
for val in $value; do

has essentially identical functionality to

for val in $(ls -1); do

That doesn't work in general. It breaks if filenames have whitespace or glob characters in them, at least. See Bash Pitfalls #1 (for f in $(ls *.mp3)). In addition, there are particular problems with using the output of ls in programs. It's only suitable for interactive use. See Why you shouldn't parse the output of ls(1).

A correct, completely safe, and much shorter and faster, alternative is:

for val in *; do

A full solution for your question is:

shopt -s nullglob
for file in Result*.RData; do
    [[ -f $file && -s $file ]] && printf '%s\n' "$file"
done >CompletedJobs.txt
  • shopt -s nullglob prevents glob patterns expanding to (what amounts to ) garbage if they don't match any files.
  • I've replaced val with the more meaningful (to me anyway) file.
  • The Result*.RData causes the loop to only process files that match that pattern.
  • I've added a -f $file test to avoid processing any non-file things (directories, fifos, ...) that might be lying around. It still allows symlinks to files through. You might not want that. You can add a ! -L $file && at the start of the test expression if you want to rule out symlinks.
  • I've replaced echo "$val" with printf '%s\n' "$val" because the original code doesn't work in general. See the accepted, and excellent, answer to Why is printf better than echo?.
  • I've moved the redirection to CompletedJobs.txt outside the loop, as suggested by @CharlesDuffy in a comment.
  • Note that this code won't work if any of the files have newlines in their names (e.g. create one with echo data > $'Result\n1.RData'). That's very uncommon, but posssible. The only way to safely store general unquoted filenames in files is to separate them with ASCII NUL characters (which can't appear in file names). To do that, replace the printf ... with printf '%s\0' "$file". That would mean that CompletedJobs.txt is no longer a text file though. It would also require modifications to any tools that read the file.

You could also do this just with find:

find . -maxdepth 1 -type f -name 'Result*.RData' -not -size 0 -printf '%P\n' >CompletedJobs.txt
  • The %P format with -printf removes the leading ./ from outputs so you get Result2.RData instead of ./Result2.RData (which find would print by default, or with the -print option).
  • Replace \n with \0 to make the output safe for any possible filename.
user_41543750

-s file True if file exists and has a size greater than zero.

ls -1 > AllFilesPresent.txt
value=$(<AllFilesPresent.txt)
rm AllFilesPresent.txt
for val in $value; do
  if [[ -s "${val}" ]]; then
    case $val in
      (Result*.RData) echo "$val" >> CompletedJobs.txt ;;
    esac
  fi
done
user_27779880
Related