OpenSolaris Build 54 is now in the wild. As of Build 53 we have an amazing powerful new feature that will ultimately become a staple of the data center: ZFS and iSCSI Integration. Its now drop dead simple to start dishing out iSCSI Targets to your network, in a way that only ZFS can provide.

If you didn’t already know, ZFS brings the functions of a filesystem and a volume manager together into perfect harmony. Now, when people hear this they probly think about the underlying zpool on which filesystems are built, but it goes further. You can create zvols, a traditional allocated block device similar to a Veritas Volume Manager Volume or LVM Logical Volume, that coexists happily with your other ZFS file systems. Because iSCSI Target LUN’s are shared network accessible block storage devices they are built on top of these zvols. The advantage of these zvols is that they come with all that ZFS has to offer filesystems including snapshots, replication, compression, etc, but adds in something special for block storage: sparse volumes, commonly refered to as “thin provisioning”. Learn more about thin provisioning with ZFS in my blog entry entitled: ZFS and Thin Provisioning.

Prior to B53 you could share a zvol via iSCSI using iscsitadm, the administrative interface to the Solaris iSCSI Target. But thats all like tedious and involves a lot of typing, therefore we can now simplify the proccess. Simply create a zvol and enable iSCSI like so:

[thumper3:/] root# zfs create -s -V 150tb splash/uber_iscsitarget
[thumper3:/] root# zfs set shareiscsi=on splash/uber_iscsitarget

Presto! With these two commands I’ve created a thinly provisioned 150TB volume and then turned “shareiscsi” on. We’re done!

Don’t believe me? Just check:

[thumper3:/] root# svcs -a | grep -i iscsi
disabled        9:04:43 svc:/network/iscsi_initiator:default
online         22:21:01 svc:/system/iscsitgt:default
[thumper3:/] root# iscsitadm list target -v
Target: splash/uber_iscsitarget
    iSCSI Name: iqn.1986-03.com.sun:02:df204646-7956-e0a9-bfc1-c47759d0d93b
    Alias: splash/uber_iscsitarget
    Connections: 0
    ACL list:
    TPGT list:
    LUN information:
        LUN: 0
            GUID: 0x0
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size:  150T
            Backing store: /dev/zvol/rdsk/splash/uber_iscsitarget
            Status: online
[thumper3:/] root# zfs get all splash/uber_iscsitarget
NAME                     PROPERTY       VALUE                  SOURCE
splash/uber_iscsitarget  type           volume                 -
splash/uber_iscsitarget  creation       Mon Dec 11  8:22 2006  -
splash/uber_iscsitarget  used           54.8K                  -
splash/uber_iscsitarget  available      15.9T                  -
splash/uber_iscsitarget  referenced     54.8K                  -
splash/uber_iscsitarget  compressratio  1.00x                  -
splash/uber_iscsitarget  reservation    none                   default
splash/uber_iscsitarget  volsize        150T                   -
splash/uber_iscsitarget  volblocksize   8K                     -
splash/uber_iscsitarget  shareiscsi     on                     local
splash/uber_iscsitarget  checksum       on                     default
splash/uber_iscsitarget  compression    off                    default
splash/uber_iscsitarget  readonly       off                    default

iSCSI really is that easy!

At this point we can go ahead and mount it on another system. In this case I’ll use a OpenSolaris B43 admin server:

[atlantis:/] root# iscsiadm list  discovery
Discovery:
        Static: disabled
        Send Targets: disabled
        iSNS: disabled
[atlantis:/] root# iscsiadm modify discovery --sendtargets enable
[atlantis:/] root# iscsiadm add discovery-address 10.10.10.10
[atlantis:/] root# svcadm enable network/iscsi_initiator
[atlantis:/] root# iscsiadm list target
Target: iqn.1986-03.com.sun:02:df204646-7956-e0a9-bfc1-c47759d0d93b
        Alias: splash/uber_iscsitarget
        TPGT: 1
        ISID: 4000002a0000
        Connections: 1
[atlantis:/] root# format
Searching for disks...done

c4t010000144F20FD1A00002A00457D16C8d0: configured with capacity of 153600.00GB

AVAILABLE DISK SELECTIONS:
...
       4. c4t010000144F20FD1A00002A00457D16C8d0 
          /scsi_vhci/disk@g010000144f20fd1a00002a00457d16c8
Specify disk (enter its number): ^D

See? Once the target is accessible from an initiator on your network simply use the disk like you world any local drive.

…but wait. We can make this even simpler by using ZFS property inheritance! By creating what I call a ZFS stub, we can turn on “shareiscsi” and then let volume created under that stub be automatically shared because they’ll inherit the “shareiscsi=on” properly. Lets go back to the begining and try this on a clean system:

[thumper3:/] root# zfs create splash/iscsi_luns
[thumper3:/] root# zfs set shareiscsi=on splash/iscsi_luns

[thumper3:/] root# iscsitadm list target
[thumper3:/] root#

[thumper3:/] root# zfs create -s -V 25g splash/iscsi_luns/vol001
[thumper3:/] root# zfs create -s -V 25g splash/iscsi_luns/vol002
[thumper3:/] root# zfs create -s -V 25g splash/iscsi_luns/vol003
[thumper3:/] root# zfs create -s -V 25g splash/iscsi_luns/vol004

[thumper3:/] root# iscsitadm list target
Target: splash/iscsi_luns/vol001
    iSCSI Name: iqn.1986-03.com.sun:02:736c2517-91e6-cf6c-d566-d48fb182e9f7
    Connections: 0
Target: splash/iscsi_luns/vol002
    iSCSI Name: iqn.1986-03.com.sun:02:78692333-1fb4-ee45-b547-ea49922ee538
    Connections: 0
Target: splash/iscsi_luns/vol003
    iSCSI Name: iqn.1986-03.com.sun:02:792505e0-a3fc-4ccf-f86d-b79f7baee006
    Connections: 0
Target: splash/iscsi_luns/vol004
    iSCSI Name: iqn.1986-03.com.sun:02:994ce0c3-d1fd-c0c9-ad8e-c4eeef462899
    Connections: 0

Now you can create iSCSI Targets by simply creating a volume. Simply brilliant.