What would be the bash command that will continuously watch jsondata folder and sync that folder with s3 when changes happen to local folder.
Now I am running this command but it does not have a -watch flag.
aws s3 sync jsondata s3://mybucket/jsondata/
In Linux, you can use incrontab utility to watch over files and folders. Trigger the script whenever you receive a new file in folder. The script will contain aws sync command.
S3 Sync is a bash script that will setup automatic syncing between your local directory and Amazon S3, utilizing the inotify linux kernel utility and s3cmd. It can also sync from your S3 bucket to a local directory via cron.
I too had the same problem and came across this stackoverflow question and figured there has to be a more convenient way to do this.
So I wrote a little npm module exactly for this purpose. it watches a folder and uploads file changes to an s3 bucket automatically. It's called s3-auto-sync
Once installed you can just run s3-auto-sync -r us-east-1 -b my-bucket -d ./my-folder. That's it.
PS: Use a bucket in the same region as you are located, this will significantly improve upload speed.
Seems like there are many options but here is one from a quick google search:
https://pypi.org/project/watchdog/
Python script that can register for filesystem notifications and take an action as a response. The examples shown at the link above can take a shell command (s3 sync, for example) that can be executed when a change in the directory is detected.
The inotify_simple project is another alternative. There are many options.