logo

0. Install Open ZFS on Ubuntu.

sudo apt install zfsutils-linux

1. Create a pool

sudo zpool create red /home/ubuntu/0
  • Creates a pool named red, with single vdev and single disk. In this case disks is a regular file. Full path needs to be provided. Otherwise in Ubuntu /dev/sdc is used.

red

2. Create a dataset

sudo zfs create red/downloads

  • Creates a dataset named downloads.

  • sudo is needed for mounting permissions because regular users can not mount to filesystem.

  • Mount point defaults if not specified.

zfslist

3. Attach a device to vdev

sudo zpool attach red /home/ubuntu/0 /home/ubuntu/1
  • The pool will "resilver".

attach

4. Detach a device from vdev

sudo zpool detach red /home/ubuntu/1
  • The pool will "resilver".

attach

5. Add a second "mirror" vdev to pool

sudo zpool add red mirror /home/ubuntu/2 /home/ubuntu/3

add vdev

6. Remove top-level vdev from pool

sudo zpool remove red mirror-1

vdev rem

Gives an error.

  • It should work with a physical disk, testing here is done with files.
  • device_removal feature is enabled.
  • Use zpool get all red to list all properties and features of pool red.
  • Use zpool upgrade red to upgrade ZFS features for pool red.
  • zpool-remove man pages

To enable the feature run

sudo zpool set feature@device_removal=enabled red

7. Destroy dataset

sudo zfs destroy red/downloads
  • Destroys dataset downloads on the pool named red.

8. Destroy pool

sudo zpool destroy red
  • Destroys the pool named red.

9. Import pool

sudo zpool import blue
  • Imports pool blue on same machine or a new machine.

import pool

  • This is useful if moving disks to a new server machine, then all is needed is ZFS installed and run the command to import the pool.

  • Another case is if the OS is reinstalled, then the pool can be imported. Given that the OS isn't installed on the pool as zroot or "root on ZFS".

10. Snapshots

zfs snapshot -r blue/backups@test
  • Creates snapshots for blue/backups and for all descendent file systems by using the -r option.
zfs list -t snapshot
  • Lists snapshots

snapshot

zfs send blue/backups@test | sudo zfs recv blue/backups/new

  • Send and receive snapshot and mount it as a new dataset at blue/backups/new.

  • A dataset is considered an independent file system, moving files across datasets is io intensive even when inside the same pool.

sudo zfs send -R zroot/usr/home@today | ssh user@irondesign.dev sudo zfs recv blue/backups/FreeBSD

  • Sends a snapshot over SSH, mounts it on backup server as a new dataset with all children datasets using option -R. sudo is needed at receiving ent to mount the dataset to fs.

  • If using incremental snapshots use option -i, and include the original snapshot and all incremental subsequent snapshots. List snapshots

11. ZFS Delegated Permissions

zpool get delegation blue
  • Checks if delegation property is on.
zfs allow blue
  • Displays Permissions on pool blue.

permissions

sudo zfs allow ubuntu create,destroy,mount,mountpoint,snapshot blue
  • Sets Permissons for user ubuntu on pool blue.
sudo zfs allow ubuntu create,destroy,mount,mountpoint,snapshot blue/ubuntu
  • Sets Permissons for user ubuntu on dataset blue/ubuntu.

12. Other commands

zfs mount
  • Shows mounted datasets with mount points.

mount

zpool iostat 5
  • Displays io stats every 5 seconds. Add option -v to include vdevs.

iostat

zfs get all
  • Displays properties and features.
zfs allow blue/downloads
  • Displays Permissions on dataset blue/downloads.

Documentation

FreeBSD specific commands

geom disk list
  • Lists disks.
smartctl -a /dev/da0
  • Displays S.M.A.R.T. info for device da0.
sudo diskinfo -v da0
  • Displays info for disk da0.
sudo smartctl --scan
  • Scans for devices.

Testing ZFS using files

for ((i=0;i<=5;++i)); do dd if=/dev/zero of=disk_$i bs=1M count=100; done

for i in $(seq 6); do dd if=/dev/zero of=disk_$i bs=1M count=100; done

  • Creates 6 empty files for testing.

  • Files need to be larger than 64MB to be used in ZFS.

sudo zpool create purpleelephants mirror /home/ubuntu/disk_0 /home/ubuntu/disk_1 /home/ubuntu/disk_2 mirror /home/ubuntu/disk_3 /home/ubuntu/disk_4 /home/ubuntu/disk_5
  • Creates a pool consisting of two mirrored vdevs with three disks each.

Root on ZFS (zroot) considerations

  • OS can be installed on a separate disk with either UFS EXT4 or ZFS (root on ZFS) fs, and have separate ZFS pools of disks for user data.

  • OS can live on a single ZFS pool along with user data on a single zroot pool consisting of however many disks plus have other ZFS pools of disks.