Rsync is a great tool for keeping files synchronized between two or more computers. However, it can use a lot of bandwidth if it’s run constantly. Here are some tips to keep rsync from using all your bandwidth:
- Use rsync only when necessary. If you only need to synchronize a few files, use the –partial option to reduce the amount of data transferred.
- Use compression when possible. Rsync can compress the data before transferring it over the network, saving on bandwidth and improving performance.
- Use a low-bandwidth connection when possible. If you’re using a low-bandwidth connection, try using rsync over SSH instead of over the network. SSH is more efficient than TCP/IP because it uses encryption to protect the data transfer. ..
The point of limiting bandwidth is to make sure your backup scripts don’t clog up the network connection between your web server and database server. Naturally, limiting the amount of bandwidth your backups are using is going to make them happen more slowly, but if you can deal with that, this is the way to do it.
Normally your rsync command will look something like this:
What you’ll want to do is use the –bwlimit parameter with a KB/second value, like this:
So if you wanted to limit transfer to around 5MB per second, you could use a command like this:
You’ll want to make sure to calculate the size of your backups, and figure out how long it’ll take to transfer them at a slower rate – it wouldn’t be very helpful if your backup script takes 12 hours to run, for example.
You can also use the ionice command to limit the amount of bandwidth your script uses, which will help keep your scripts from using too much disk I/O. It’s not quite as useful since you can’t set absolute numbers like you can with the bwlimit parameter, but it’s another thing you can consider.