Adding a loop within python sub-process

Question

I need some help! I have a script that loops through all of the files in a directory and runs a subprocess to use a command line tool (fimo) on each file. Fimo is a tool used to search genetic sequences for specific subsequences, or motifs. As my code exists now, only 1 motif is being searched for ("/home/MA0074.1.meme").

I would like to change this code so that it loops through more than 1 motif. I am not sure how to add a loop into the subprocess, or how a variable within a subprocess might work (something like another {f}?), and I'm having trouble finding information.

Additional motifs could be like:

"/home/MA0796.1.meme"
"/home/MA0113.3.meme"

etc.

Here's the code as it exists now:

import subprocess
import sys
from subprocess import Popen, PIPE
import os
import os.path

os.chdir("/home/meme/src")
i = 0

path_of_the_directory= "/home/fastas_no_canonical"
for filename in os.listdir(path_of_the_directory):
    f = os.path.join(path_of_the_directory,filename)
    print(filename)
    i= i+1
    placeHolder = str(os.path.basename(filename))
    if os.path.isfile(f):
        result = subprocess.run(f'./fimo --oc "/home/fimos/%s" --verbosity 1 --thresh 1.0E-4 "/home/MA0074.1.meme" "{f}"'%(placeHolder), shell=True)
        print(f)

EDIT: I have added a second for loop to my code and fixed the f-strings like so:


i = 0

path_of_the_directory= "/home/fastas_no_canonical"
path_of_memes= "/home/test_memes"
for filename in os.listdir(path_of_the_directory):
    for meme in os.listdir(path_of_memes):
        f = os.path.join(path_of_the_directory,filename)
        m = os.path.join(path_of_memes,meme)
        i= i+1
        if os.path.isfile(f):
            result = subprocess.run(f'./fimo --oc "/home/test_memes/junk/{filename}/{meme}" --verbosity 1 --thresh 1.0E-4 "{m}" "{f}"', shell=True)
            print(f)
            print(m)

It is working, but only for the first file. The output contains a folder with data for each motif search for the first input file, but then freezes. I checked with print(f) and print(m) that the file names are right, so this is how I can tell that it freezes after the first file.

Does anyone see an error in my loop that might be causing this? Do I need another iterator?

user_1440565 · Answer

I am not sure how to add a loop into the subprocess, or how a variable within a subprocess might work (something like another {f}?)

You have some of the right idea here, but this is the incorrect way to think about it. You don't create a variable "within a subprocess". Instead, you have a variable in your python code that stores the path to a motif. You add the name in that variable to the string which you pass to subprocess.run(). You can even make a loop that iterates over a list of these paths. In pseudocode it might look something like this:

for each dna sequence file
   for each motif
       search the dna sequence for the motif

As a general rule, describing the steps to solve a problem in words in this way is a great place to start. I'll leave it to the reader to translate this into code.

On a side note, either use f'' or ''%(), but not both.

Adding a loop within python sub-process

1 Answers