There are a lot of core concepts to grasp with ZFS. Frankly, power comes at the price of a learning curve. While ZFS makes storage simpler than ever, a solid grasp of the core concepts is really critical to efficient usage.
Anytime someone says “multiple filesystems” I cringe. There is no faster way to lead yourself down a confused road than to think of the zfs create command as creating a new filesystem. You’re creating a new “dataset”. Lets say it a couple times to let it sink in: dataset dataset dataset. Datasets should be thought of, first and foremost as organizational boundaries. Data demarcation if you please. With in a large structure typically called a filesystem we sub-divide the structure into independently or cooperatively controllable entities… datasets!
To ZFS there are two types of datasets: filesystem datasets and volume datasets. Filesystem datasets store file data in the traditional manner, while volume datasets are block allocations which can be used to store non-ZFS filesystems (UFS, VxFS, etc), raw data allocations (used by apps like Oracle), or offered up via technologies like iSCSI.
Each dataset has properties associated with it. A filesystem dataset has properties such as atime (whether or not to update timestamps on the file when accessed; this creates lots of additional IO you commonly don’t need), compression, mount point, etc. A volume dataset has fewer properties, such as compression, readonly, and shareiscsi.
Properties can be inherited from a parent dataset to its children, and if you wish they can be overridden in special instances. For instance, if I want to control the quota of each user directory independantly each will need an individual dataset, however if I want all of them to have a default quota of 10GB, disable atime, and enable compression, then I can set those properties on the parent dataset and each child will recieve those properties.
When I create a dataset that will not contain files but simply acts as a parent for a bunch of children datasets I refer to it as a “stub”. For instance, my pool might be called “bigpool”, my stub dataset would be “bigpool/home”, and each user directory would be a child of that stub dataset: “bigpool/home/userA”, “bigpool/home/userB”, etc.
Of course, by default the mount point would be “/bigpool/home/userA”, for instance, and thats just ugly. Therefore on my “home” stub I’d set (after disabling the the automounter; svcadm disable autofs) the mountpoint to “/home” ala zfs set mountpoint=/home bigpool/home.
If you use the old “filesystem” terminology you can quickly get confused, therefore thinking in “datasets” gives your brain something new to better attach these ideas of properties and propagation of them far more easily.
Stub datasets for the win!