How to set InputStream content Length

user_1364073 0
Viewed 43520

I am uploading files to Amazon S3 bucket. The files are being uploaded but i get the following Warning.

WARNING: No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.

So I added the following line to my code

metaData.setContentLength(IOUtils.toByteArray(input).length);

but then i got the following message. I don't even know if it is a warning or what.

Data read has a different length than the expected: dataLength=0; expectedLength=111992; includeSkipped=false; in.getClass()=class sun.net.httpserver.FixedLengthInputStream; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0

How can i set contentLength to the metaData of InputSteam? Any help would be greatly appreciated.

amazon-web-services inputstream metadata amazon-s3
3 Answers

Note that by using a ByteBuffer you simply do manually what the AWS SDK already did for you automatically! It still buffers the entire stream into memory and is as good as the original solution which produces the warning from the SDK.

You can only get rid of the memory-problem if you have another way to know the length of the stream, for instance, when you create the stream from a file:

void uploadFile(String bucketName, File file) {
    try (final InputStream stream = new FileInputStream(file)) {
        ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentLength(file.length());
        s3client.putObject(
                new PutObjectRequest(bucketName, file.getName(), stream, metadata)
        );
    }
}
user_6764270

Breaking News! AWS SDK 2.0 has built-in support for uploading files:

        s3client.putObject(
                (builder) -> builder.bucket(myBucket).key(file.getName()),
                RequestBody.fromFile(file)
        );

There's also RequestBody methods for taking Strings or Buffers which automatically and efficiently set Content-Length. Only when you have another kind of InputStream, you still need to provide the length yourself – however that case should be more rare now with all the other options available.

user_6764270
Related