MongoDB backup plan

user_437763 0
Viewed 13762

I want to switch from MySQL to MongoDB but great data losses (more than 1 hour) are not acceptable for me.

I need to have 3 backup plans:

  1. Hourly backup plan. Data is flushed to disk every X minutes and if something wrong with the server I shall be sure that after reboot it will have all data at least for an hour ago. Can I configure it?

  2. Daily backup plan. Data is synced to backup disk every day so even if server explodes I can recover data for yesterday in some hours. Should I use fsync, master-slave or something else? I would like to have minimal traffic so ideally only changes will be sent.

  3. Weekly backup plan. Data is synced to second backup disk so if both server and first backup disk explode I have at least data for last week. Here this is the question of reliability so it's ok to send all data via network.

How can I do it?

database-backups mongodb backup
5 Answers

Try this backup script if you want to create a backup from slave mongodb database to S3.

  1. DB host (secondary preferred as to avoid impacting primary performance)

    HOST='SomeHost/mongodbtest-slave'

  2. DB name DBNAME=***

  3. S3 bucket name BUCKET=*-backup

  4. Linux user account USER=ubuntu

  5. Current time TIME=/bin/date +%d-%m-%Y-%T

  6. Password PASSWORD=somePassword#!2*1

  7. Username USERNAME=someUsername

  8. Backup directory DEST=/home/ubuntu/tmp

  9. Tar file of backup directory TAR=$DEST/../$TIME.tar

  10. Create backup dir (-p to avoid warning if already exists) /bin/mkdir -p $DEST

  11. Log echo "Backing up $HOST/$DBNAME to s3://$BUCKET/ on $TIME";

  12. Dump from mongodb host into backup directory

mongodump --port 27017 -d DBNAME -u USERNAME -p PASSWORD -o $DEST

  1. Create tar of backup directory /bin/tar cvf $TAR -C $DEST .

  2. Upload tar to s3 /usr/bin/aws s3 cp $TAR s3://$BUCKET/

  3. Remove tar file locally /bin/rm -f $TAR

  4. Remove backup directory /bin/rm -rf $DEST

All done echo "Backup available at https://s3.amazonaws.com/$BUCKET/$TIME.tar

You can use the steps above put them in a shell executable file and execute this at any interval using crontab commands.

user_102736440

On the first point.

MongoDB has a term like 'durable write operation'. If journaling is enabled, you may only lose data that has not been writed in the log. This is a very small amount of time (100 milliseconds by default)

On the second and third points.

You can set master slave replication, but this will not protect you from data errors (for example, if important data is accidentally deleted). Therefore, you need to provide regular backups one way or another.

There are several approaches here:

  1. LVM snapshot is a good solution if your filesystem supports it and the journaling is enabled in your mongoDB. For more details see here
  2. mongodump -You can create a shell script that will run scheduled backups via cron and send them to storage. Here is an example of a good script
  3. Backup as a Service. There are many solutions to make the backup for you.
user_92107270
Related