I have log files generated by my application running on EC2. I want to first upload the log file to a bucket in S3. I then run a map reduce job and I want to then download the output file back to EC2.
What is the best way for me to upload and download files from EC2 and S3? Is it possible to do automate the process using Shell script on EC2, given that I'm using a linux instance? Also, would I need to modify access permission on the s3 bucket?