Using Rsync to backup data

Sync

Send incremental file list, delete files at destination if files don't exist at origin, preserve hardlinks, progress, partial transfer in case of network interrupt, compress, verbose.

This command will essentially sync or mirror the local directory to the remote directory, when the command runs again newly added files will be uploaded and files that are deleted in local directory will be removed from remote directory also.

Transfer contents of directory disks.

rsync -avzHP --delete ~/disks/ iron@irondesign.dev:/home/iron/disks

If you omit the forward slash at the end of disks/ Rsync will transfer the directory disks and it's contents to a new directory on remote host named disk.

rsync -avzHP --delete ~/disks iron@irondesign.dev:/home/iron/disks

End result is /home/iron/disks/disks.

Relative path

Relative path flag -P will copy full path /etc/nginx/sites-enabled/default and create if it doesn't exist on remote dir /blue/backups/.

rsync -aP --relative /etc/nginx/sites-enabled/default me@irondesign.dev:/blue/backups/wg/

End result is /blue/backups/etc/nginx/sites-enabled/default.

Delete extraneous files

rsync -avzH --delete --existing --ignore-existing --ignore-errors data databases user23@10.0.0.23:/home/user23

  --existing, --ignore-non-existing
          This tells rsync to skip creating files (including  directories)
          that  do  not  exist  yet on the destination.  If this option is
          combined with the --ignore-existing option,  no  files  will  be
          updated  (which  can  be  useful if all you want to do is delete
          extraneous files).

• If there are any errors during an rsync scync, rsync will not properly delete the files it should have, even if you used --delete, --delete-after, or --delete-before.

• Using rsync to only delete extraneous files

Flags

-a --archive This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything. Note however that -a does not preserve hardlinks, because finding multiply-linked files is expensive. You must separately specify -H.

-z --compress With this option, rsync compresses the file data as it is sent to the destination machine, which reduces the amount of data being transmitted -- something that is useful over a slow connection.

-v --verbose This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single -v will give you information about what files are being transferred and a brief summary at the end. Two -v flags will give you information on what files are being skipped and slightly more information at the end.

-P combines the flags --progress and --partial

--progress This option tells rsync to print information showing the progress of the transfer. This gives a bored user something to watch.

--partial By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances it is more desirable to keep partially transferred files. Using the --partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster.

-t, --times preserve modification times

-H, --hard-links This tells rsync to look for hard-linked files in the transfer and link together the corresponding files on the receiving side. Without this option, hard-linked files in the transfer are treated as though they were separate files.

When you are updating a non-empty destination, this option only ensures that files that are hard-linked together on the source are hard-linked together on the destination. It does NOT currently endeavor to break already existing hard links on the destination that do not exist between the source files. Note, however, that if one or more extra-linked files have content changes, they will become unlinked when updated (assuming you are not using the --inplace option).

Note that rsync can only detect hard links between files that are inside the transfer set. If rsync updates a file that has extra hard-link connections to files outside the transfer, that linkage will be broken. If you are tempted to use the --inplace option to avoid this breakage, be very careful that you know how your files are being updated so that you are certain that no unintended changes happen due to lingering hard links (and see the --inplace option for more caveats).

If incremental recursion is active (see --recursive), rsync may transfer a missing hard-linked file before it finds that another link for that contents exists elsewhere in the hierarchy. This does not affect the accuracy of the transfer, just its efficiency. One way to avoid this is to disable incremental recursion using the --no-inc-recursive option.

--delete Delete files in remote directory that don't exist in local directories.