Is it possible download file with Python ftplib at the same time as retrieving file list information from the list function?

user_7410588 0
Viewed 91

I have a scenario that looks something like below. My goal is to print out the lines of the file that I'm retrieving from the Python ftplib retrlines("LIST") function. Any help is appreciated.

class FTP:

    def __init__(self, hostName, userName, passWord, encoding=None):
        self.ftp = ftplib.FTP(host=hostName, user=userName, passwd=passWord)
        self.defaultEncoding = self.ftp.encoding
        self.ftp.encoding = encoding or self.ftp.encoding
    
    def writeTest(self, fileName, func):
        self.ftp.retrlines(f'RETR {fileName}', func)

    def getFileInformation(self, func=None):

        encoding = self.ftp.encoding
        self.ftp.encoding = self.defaultEncoding
        self.ftp.retrlines('LIST', lambda row: func(self._returnFileInformation(row)))
        self.ftp.encoding = encoding


class FileProcessor:

    def __init__(self, processingFunction: Callable, output: Callable):
        self.processingFunction = processingFunction
        self.output = output
        self.fileCounter = 0

    def processFile(self, file: FileInformation):
        if self.processingFunction(file):
            self.output(file)
            self.fileCounter += 1

def __len__(self):
    return self.fileCounter

lastReadTime = datetime(2020, 10, 29, 0, 0, 0)
ftp = FTP(hostName=hostName, userName=userName, passWord=passWord, encoding='utf-16')
processor = FileProcessor(lambda file: file.timeStamp > lastReadTime, lambda file: ftp.writeTest(file.name, lambda line: print(line)))
ftp.getFileInformation(processor.processFile)

The getFileInformation just returns the values from the retrlines LIST function in a nice formatted class that has the name, size, etc. I'm just trying to print the lines of the files, as I'm getting the file names from retrlines('LIST'). If I retrieve the file names first, and then process the files after, I have no problem. If I try to do it all at once I get an error that looks something like this:

enter image description here

ftplib ftp python
1 Answers

You cannot download files, while you are still downloading a directory listing using the same connection. That's not possible with the FTP protocol, no matter what FTP library you are using. In terms of the ftplib API: you cannot call back to the FTP class (you cannot call FTP.retrlines('RETR ...')), while another method (FTP.retrlines('LIST ...')) is still executing.

Either:

  • Open two connections, one for the listing and one for file downloads.

  • Or stick with "retrieve the file names first, and then process the files after" – I do not see what's wrong about it.

user_8508480
Related