AWS CLI for S3 Select

user_13583000 0
Viewed 4471

I have the following code, which is used to run a SQL query on a keyfile, located in a S3 bucket. This runs perfectly. My question is, I do not wish to have the output written over to an output file. Could I see the output on the screen (my preference #1)? If not, what about an ability to append to the output file, rather than over-write it (my preference #2). I am using the AWS-CLI binaries to run this query. If there is another way, I am happy to try (as long as it is within bash)

aws s3api select-object-content \
    --bucket "project2" \
    --key keyfile1 \
    --expression "SELECT * FROM s3object s where Lower(s._1) = 'email@search.com'" \
    --expression-type 'SQL' \
    --input-serialization '{"CSV": {"FieldDelimiter": ":"}, "CompressionType": "GZIP"}' \
    --output-serialization '{"CSV": {"FieldDelimiter": ":"}}' "OutputFile"
amazon-s3-select amazon-web-services amazon-s3 sql
3 Answers

Of course, you can use AWS CLI to do this since stdout is just a special file in linux.

aws s3api select-object-content \
--bucket "project2" \
--key keyfile1 \
--expression "SELECT * FROM s3object s where Lower(s._1) = 'email@search.com'" \
--expression-type 'SQL' \
--input-serialization '{"CSV": {"FieldDelimiter": ":"}, "CompressionType": "GZIP"}' \
--output-serialization '{"CSV": {"FieldDelimiter": ":"}}' /dev/stdout

Note the /dev/stdout in the end.

user_106924930

The AWS CLI does not offer such options.

However, you are welcome to instead call it via an AWS SDK of your choice.

For example, in the boto3 Python SDK, there is a select_object_content() function that returns the data as a stream. You can then read, manipulate, print or save it however you wish.

user_1747770

I think it opens /dev/stdout twice causing kaos.

user_146459080
Related