I've a requirement wherein I have to implement a script to transfer files from a remote Solaris server to AWS S3 bucket. I should be running this script on my AWS EC2 instance for every 24 hours such that it connects to the remote Solaris server and transfer the files to S3 bucket.
Can anyone guide me how to implement this? Thanks.
Simplest way is that You need to first use scp to copy file from remote server to local AWS server; and then use AWS CLI commond "aws s3 cp" to copy this file to s3 bucket. This script should be called every 24 hours using crontab.
Change below the mynightbackup to the name of your AWS S3 bucket. Put in the include_dirs.txt the path to those directories you want to backup. Run this manually like this:
nohup ./file_server_backup_to_aws_s3.sh 2>&1 &
Or put file_server_backup_to_aws_s3.sh in the CRON to run every 24 hours.
This writes the log to the log file named: file_server_backup_to_aws_s3.log And you can review the output of the script in the nohup.out. The sync on the aws command line will sync those objects, similar to how rsync works.
#!/bin/bash
# Program Name: file_server_backup_to_aws_s3.sh
# Purpose: Upload files to AWS S3 remote backup
#
# Usage:
# nohup ./file_server_backup_to_aws_s3.sh 2>&1 &
echo -e "\nstart:" `date`
LOG="file_server_backup_to_aws_s3.log"
rm -rf $LOG
for SOURCE in `cat include_dirs.txt`
do
aws s3 sync $SOURCE "s3://mynightbackup"$SOURCE"/" --delete --output text >>$LOG
done
echo -e "end:" `date`