Yesterday morning, I received a panicked call from a developer:
"I accidentally filled up the storage, and now I can't perform any operations! My ZFS pool is full!"
I immediately reassured them because I had anticipated this kind of issue. One of the things I almost always do when managing ZFS file systems is to reserve space in a specially created dataset.
This is because ZFS, like all CoW (Copy-on-Write) file systems, can find itself unable to free up space when completely full. By using reserved space, I can always free it up and delete other data, restoring the system to normal operations.
To reserve space, simply create a dataset and assign it a reserved size. Of course, this dataset should not be used for anything else; otherwise, the entire purpose would be defeated.
To create it and reserve space, you only need two simple commands. For example:
zfs create zroot/reserved
zfs set reservation=5G zroot/reserved
This creates the dataset and assigns it 5 GB of reserved space.
Here’s the situation before the operation:
zfs list zroot
NAME USED AVAIL REFER MOUNTPOINT
zroot 3.02G 109G 96K /zroot
And here’s the situation after:
zfs list zroot
NAME USED AVAIL REFER MOUNTPOINT
zroot 8.02G 104G 96K /zroot
As you can see, the 5 GB are removed from the available space and marked as used, but they are actually empty.
In case of a full file system, you can delete this dataset (or reduce its size) to return to normal file system operation.
Even with this technique, I still recommend not filling ZFS pools beyond 80% of their capacity, as performance degrades significantly past that point.