Adding test_ in front of a file name with path

user_13595636 0
Viewed 69

I have a list of files stored in a text file, and if a Python file is found in that list. I want to the corresponding test file using Pytest.

My file looks like this:

/folder1/file1.txt
/folder1/file2.jpg
/folder1/file3.md
/folder1/file4.py
/folder1/folder2/file5.py

When 4th/5th files are found, I want to run the command pytest like:

pytest /folder1/test_file4.py
pytest /folder1/folder2/test_file5.py

Currently, I am using this command:

cat /workspace/filelist.txt | while read line; do if [[ $$line == *.py ]]; then exec "pytest test_$${line}"; fi; done;

which is not working correctly, as I have file path in the text as well. Any idea how to implement this?

pytest shell unix bash
3 Answers

Using Bash's variable substring removal to add the test_. One-liner:

$ while read line; do if [[ $line == *.py ]]; then echo "pytest ${line%/*}/test_${line##*/}"; fi; done < file

In more readable form:

while read line
do 
  if [[ $line == *.py ]]
  then 
    echo "pytest ${line%/*}/test_${line##*/}"
  fi
done < file

Output:

pytest /folder1/test_file4.py
pytest /folder1/folder2/test_file5.py

Don't know anything about the Google Cloudbuild so I'll let you experiment with the double dollar signs.

Update:

In case there are files already with test_ prefix, use this bash script that utilizes extglob in variable substring removal:

shopt -s extglob                                           # notice
while read line
do
    if [[ $line == *.py ]]
    then
        echo "pytest ${line%/*}/test_${line##*/?(test_)}"  # notice
    fi
done < file
user_41623560

You may want to focus on lines that ends with ".py" string You can achieve that using grep combined with a regex so you can figure out if a line ends with .py - that eliminates the if statement.

IFS=$'\n'
for file in $(cat /workspace/filelist.txt|grep '\.py$');do pytest $file;done
user_25509760

You can easily refactor all your conditions into a simple sed script. This also gets rid of the useless cat and the similarly useless exec.

sed -n 's%[^/]*\.py$%test_&%p' /workspace/filelist.txt |
xargs -n 1 pytest

The regular expression matches anything after the last slash, which means the entire line if there is no slash; we include the .py suffix to make sure this only matches those files.

The pipe to xargs is a common way to convert standard input into command-line arguments. The -n 1 says to pass one argument at a time, rather than as many as possible. (Maybe pytest allows you to specify many tests; then, you can take out the -n 1 and let xargs pass in as many as it can fit.)

If you want to avoid adding the test_ prefix to files which already have it, one solution is to break up the sed script into two separate actions:

sed -n '/test_[^/]*\.py/p;t;s%[^/]*\.py$%test_&%p' /workspace/filelist.txt |
xargs -n 1 pytest

The first p simply prints the matches verbatim; the t says if that matched, skip the rest of the script for this input.

(MacOS / BSD sed will want a newline instead of a semicolon after the t command.)

sed is arguably a bit of a read-only language; this is already pressing towards the boundary where perhaps you would rewrite this in Awk instead.

user_8741880
Related