Downloaded an Amazon S3 file doesn't contain data

user_16699020 0
Viewed 341

I'm writing a service to download a file in an Amazon S3 bucket, but I'm receiving nothing but an empty JSON object

Vue | Axios -

downloadLocations() {
  axios.get("http://localhost:8080/api/v1/targetLocation/downloadSearchData")
      .then((response) => {
        const  content = new Blob([JSON.stringify(response.data)],{ type: 'text/plain;charset=utf-8' })
        const fileName = `test.txt`
        saveAs(content, fileName)
      }, (error) => {
        console.log(error);
      });
}

Java | Springboot Service -

public ByteArrayOutputStream downloadSearchData() throws IOException {
    BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
    AmazonS3 s3client = AmazonS3ClientBuilder
            .standard()
            .withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
            .withRegion(awsRegion)
            .build();

    S3Object s3Object = s3client.getObject("downloadable-cases", "7863784198_2021-08-16T13_30_06.690Z.json");
    InputStream is = s3Object.getObjectContent();
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    int len;
    byte[] buffer = new byte[4096];
    while ((len = is.read(buffer, 0, buffer.length)) != -1) {
        outputStream.write(buffer, 0, len);
    }
    return outputStream;
}

THIS works, accept it downloads it to the project, and not my machine (which is the whole point) -

public void downloadSearchData() throws IOException {
    BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
    AmazonS3 s3client = AmazonS3ClientBuilder
            .standard()
            .withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
            .withRegion(awsRegion)
            .build();

    S3Object s3object = s3client.getObject(
            "downloadable-cases", "7863784198_2021-08-16T13_30_06.690Z.json"
    );
    S3ObjectInputStream inputStream = s3object.getObjectContent();
    FileUtils.copyInputStreamToFile(inputStream, new File("hello.txt"));
}
spring-boot vue.js amazon-web-services download java
2 Answers

A few comments here:

1 - you are using the older V1 Amazon S3 Java API. You should consider moving to V2 as Amazon recommends using over V1.

The AWS SDK for Java 2.x is a major rewrite of the version 1.x code base. It’s built on top of Java 8+ and adds several frequently requested features. These include support for non-blocking I/O and the ability to plug in a different HTTP implementation at run time.

See the Amazon SDK for Java V2 Developer Guide

2 - when using Amazon Java V2 APIs, you do not need to hard code your creds in the code. To work with V2 and creds, see: Using credentials.

As you are using Spring BOOT app, here is an example that uses Amazon S3 Java V2 API and Spring BOOT. This app lets you download an object located in an S3 bucket to your browser.

enter image description here

The downloaded Object is valid and i can open it and view it (its not empty or corrupt):

enter image description here

Amazon S3 Java V2 code to get an object from a bucket is:

// Get the byte[] from this AWS S3 object.
public byte[] getObjectBytes (String bucketName, String keyName) {

    s3 = getClient();

    try {
        GetObjectRequest objectRequest = GetObjectRequest
                .builder()
                .key(keyName)
                .bucket(bucketName)
                .build();
        
        ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
        byte[] data = objectBytes.asByteArray();
        return data;

    } catch (S3Exception e) {
        System.err.println(e.awsErrorDetails().errorMessage());
        System.exit(1);
    }
    return null;
}

See this developer article that shows how to work with Amazon S3 Java V2 API and other services within a Spring Boot app:

Creating an example AWS photo analyzer application using the AWS SDK for Java

This link guides you through creating this Spring app....

enter image description here

user_14355430

It looks like you have followed the official AWS example "Download an Object" so your code looks correct. You forgot to close the input stream by calling is.close() at the end, however it's unlikely that this is the problem.

You can simplify your code by using try-with-resource block and InputStream.transferTo() method:

var s3Object = s3client.getObject("downloadable-cases", "7863784198_2021-08-16T13_30_06.690Z.json");
var out = new ByteArrayOutputStream();
try (var in = s3Object.getObjectContent()) {
  in.transferTo(out);
}
return out;

Perhaps the S3 file has no content, if you download it using AWS Console will it show any content? If it works from AWS Console you should use a debugger to see what happens in your code.

user_16025550
Related