Flask file threading 'ValueError: I/O operation on closed file'

user_14855422 0
Viewed 265

I am building an API with Flask. When a user uploads a file, it should asynchronously upload the file to an AWS s3 bucket. This is done by threading this task, while the user receives a confirmation that the request was reveiced. (In this case, the files are assessed as low value and it thus not really matters if the thread explodes.) I use a Flaks request to save the file as a variable, which I pass to the threaded function which handles the files. However, when trying to access the file in this handler, I get an 'ValueError: I/O operation on closed file'. What am I doing wrong and how can I solve this? For simplicity, both functions are completely stripped down to the core without error handling.

Flask app route

@app.route('/file/upload', methods=['POST'])
def post_upload_file():
    return operator(receive_file)

Called function

def receive_file():
    if 'file' in request.files and request.files['file'] != '':
        x_file = request.files['file']

        if allowed_file(x_file.filename):
            fileid = shortuuid.uuid()
            extension = x_file.filename.rsplit('.', 1)[1].lower()
            newfilename = fileid + '.' + extension

            Thread(target = handle_file, args=(fileid, x_file, newfilename)).start()

            return {'fileid': fileid}

Threated handler

def handle_file(fileid, x_file, name=None, bucket='XXX'):

    name = "file/"+name
    s3_client = boto3.client('s3')
    s3_client.upload_fileobj(x_file, bucket, name)
flask multithreading python
1 Answers

The problem was that the file location was not existent anymore at the start of the called handle_file function, since the called function was threaded from the main thread.

Besides, I think it was probably not the best idea to pass large files around in the memory of the API server. I have solved the problem by storing the incoming files in the server space, fetching the location and passing that to the handle_file function. This retrieves the file from the storage and uploads it to the bucket.

user_148554220
Related