Using Rsync to backup data
Sync
Send incremental file list, delete files at destination if files don't exist at origin, preserve hardlinks, progress, partial transfer in case of network interrupt, compress, verbose.
This command will essentially sync or mirror the local directory to the remote directory, when the command runs again newly added files will be uploaded and files that are deleted in local directory will be removed from remote directory also.
Transfer contents of directory disks
.
rsync -avzHP --delete ~/disks/ iron@irondesign.dev:/home/iron/disks
If you omit the forward slash at the end of disks/
Rsync will transfer the directory disks
and it's contents to a new directory on remote host named disk
.
rsync -avzHP --delete ~/disks iron@irondesign.dev:/home/iron/disks
End result is /home/iron/disks/disks
.
Relative path
Relative path flag -P
will copy full path /etc/nginx/sites-enabled/default
and create if it doesn't exist on remote dir /blue/backups/
.
rsync -aP --relative /etc/nginx/sites-enabled/default me@irondesign.dev:/blue/backups/wg/
End result is /blue/backups/etc/nginx/sites-enabled/default
.
Delete extraneous files
rsync -avzH --delete --existing --ignore-existing --ignore-errors data databases user23@10.0.0.23:/home/user23
--existing, --ignore-non-existing
This tells rsync to skip creating files (including directories)
that do not exist yet on the destination. If this option is
combined with the --ignore-existing option, no files will be
updated (which can be useful if all you want to do is delete
extraneous files).
Flags
-a --archive
This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything. Note however that -a does not preserve hardlinks, because finding multiply-linked files is expensive. You must separately specify -H.
-z --compress
With this option, rsync compresses the file data as it is sent to the destination machine, which reduces the
amount of data being transmitted -- something that is useful over a slow connection.
-v --verbose
This option increases the amount of information you are given during the transfer.
By default, rsync works silently.
A single -v will give you information about what files are being transferred
and a brief summary at the end.
Two -v flags will give you information on what files are being skipped and
slightly more information at the end.
-P
combines the flags --progress
and --partial
--progress
This option tells rsync to print information showing the progress of the transfer.
This gives a bored user something to watch.
--partial
By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances
it is more desirable to keep partially transferred files. Using the --partial option tells rsync to keep the partial
file which should make a subsequent transfer of the rest of the file much faster.
-t, --times
preserve modification times
-H, --hard-links
This tells rsync to look for hard-linked files in the transfer and link together the corresponding files on the receiving side. Without this option, hard-linked files in the transfer are treated as though they were separate files.
When you are updating a non-empty destination, this option only ensures that files that are hard-linked together on the source are hard-linked together on the destination. It does NOT currently endeavor to break already existing hard links on the destination that do not exist between the source files. Note, however, that if one or more extra-linked files have content changes, they will become unlinked when updated (assuming you are not using the --inplace option).
Note that rsync can only detect hard links between files that are inside the transfer set. If rsync updates a file that has extra hard-link connections to files outside the transfer, that linkage will be broken. If you are tempted to use the --inplace option to avoid this breakage, be very careful that you know how your files are being updated so that you are certain that no unintended changes happen due to lingering hard links (and see the --inplace option for more caveats).
If incremental recursion is active (see --recursive), rsync may transfer a missing hard-linked file before it finds that another link for that contents exists elsewhere in the hierarchy. This does not affect the accuracy of the transfer, just its efficiency. One way to avoid this is to disable incremental recursion using the --no-inc-recursive option.
--delete
Delete files in remote directory that don't exist in local directories.