Reading N bytes from AWS s3 file does not read N bytes

Question

I am trying to read N bytes from my file in AWS S3 bucket and write it into buffer, however when I try to do so, the returned amount of bytes is not N, but much much less and after multiple call it differs.

What I am doing is:

public int readData(byte[] data, int doff, long offset, int length, String bucketName, String fileName) throws IOException {
    GetObjectRequest rangeObjectRequest = new GetObjectRequest(bucketName, fileName)
            .withRange(offset, length);

    S3Object objectPortion = S3_CLIENT.getObject(rangeObjectRequest);

    return objectPortion.getObjectContent().read(data, doff, length);
}

when offset = 0 and length = 32752

objectPortion.getObjectContent().read(data, doff, length);

returns 16360 while data size is 32820 which is enough for all data i want.

Why is this happening? Why isn't it returning all data?

The size of file i am trying to download is around 69kb

user_750510 · Answer

Use this snippet to read the whole content:

try (S3ObjectInputStream is = objectPortion.getObjectContent()) {
    byte[] buf = new byte[4096];
    int read = 0;

    while ((read = is.read(buf)) >= 0) {
        // use buf / read values
    }
} catch (Exception e) {
    …
}

Reading N bytes from AWS s3 file does not read N bytes

1 Answers