S3ResponseError: 400 Bad Request during multi part upload using boto

user_8704316 0
Viewed 447

I am trying to use boto to upload a large file using multi part upload.

Here's my code

import math, os
import boto.s3.connection
import boto
from filechunkio import FileChunkIO

# Connect to S3
c = boto.connect_s3()
b = c.get_bucket('bucketA')

# Get file info
source_path = './A/tmp1/'
source_size = os.stat(source_path).st_size

# Create a multipart upload request
mp = b.initiate_multipart_upload(os.path.basename(source_path))

# Use a chunk size of 50 MiB (feel free to change this)
chunk_size = 52428800
chunk_count = int(math.ceil(source_size / float(chunk_size)))

# Send the file parts, using FileChunkIO to create a file-like object
# that points to a certain byte range within the original file. We
# set bytes to never exceed the original file size.
for i in range(chunk_count):
    offset = chunk_size * i
    bytes = min(chunk_size, source_size - offset)
    with FileChunkIO(source_path, 'r', offset=offset, bytes=bytes) as fp:
        mp.upload_part_from_file(fp, part_num=i + 1)

# Finish the upload
mp.complete_upload()

My error stack is this 

---------------------------------------------------------------------------
S3ResponseError                           Traceback (most recent call last)
<ipython-input-9-2cf74dccfef2> in <module>
     22 
     23 # Create a multipart upload request
---> 24 mp = b.initiate_multipart_upload(os.path.basename(source_path))
     25 
     26 # Use a chunk size of 50 MiB (feel free to change this)

/usr/local/lib/python3.7/site-packages/boto/s3/bucket.py in initiate_multipart_upload(self, key_name, headers, reduced_redundancy, metadata, encrypt_key, policy)
   1766         else:
   1767             raise self.connection.provider.storage_response_error(
-> 1768                 response.status, response.reason, body)
   1769 
   1770     def complete_multipart_upload(self, key_name, upload_id,

S3ResponseError: S3ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message>Request Content-Type is not multipart/form-data</Message></Error>

Where exactly do I need to set the Request Content-Type? I'm using boto due to certain dependency restrictions, will not be possible for me to upgrade to boto3.

Any pointers?

boto amazon-web-services amazon-s3 python
1 Answers

The docs say the first argument is a key_name, ie the name of the file when it's available in the bucket.

It looks like you're passing in the name of a local directory './A/tmp1/' and not a large file. I don't know if this is allowed or not. The S3 docs on key names aren't super clear.

The other part of the the docs say which might be useful say the initiate_multipart_upload function takes a headers dictionary as an argument. These are standard http headers and a Content-Type header is a standard http header: source.

user_77455060
Related