The OpenSolaris COMSTAR project is one of the cornerstones of Sun’s Open Storage initiative. Quite simply, it uniquely separates and abstracts back-end storage from the means by which you allow access to it. To be more specific, through COMSTAR’s SMTF I can allocate some raw disk and then provide it to my SAN through an FC Target or iSCSI Target or SAS Target or you name it. Just think about this… you can now, with 100% open source software, turn a cheap white-box PC with a bunch of cheap SATA disks and a Fibre Channel HBA off eBay into what appears to a SAN to be an enterprise grade Fibre Channel array!

COMSTAR is short for “Common Multiprotocol SCSI Target”. It’s core is the “SCSI Target Mode Framework” (STMF). The STMF acts like a hub into which you plug Logical Unit Providers (LU… disk, allocated space on some device, tape, etc) and Port Providers (Target Implementations; think “port” as in Server Daemon). This abstraction is interesting because it means I could potentially take one allocation of disk and provide it as BOTH an iSCSI Target AND a Fibre Channel LUN!

Now, I’ll be honest, I think the concept of turning a commodity server into a Fibre Channel array is really nifty and all, but I’m not gonna do it… I don’t care. But iSCSI I most certainly do care about. It came to my attention this morning that its already available and been in development for some time, hiding as the iSCSI Extensions for Remote DMA (iSER) Project. This project is not just extentions… it is the COMSTAR iSCSI Port Provider itself, and it includes iSER support. If you’ve been waiting for the new iSCSI Target Implementation that will utilize COMSTAR, this is it!

Lets dive into this on a high level; in future blog entries I’ll dig into it much more deeply.

To get started, go to the poorly named iSER Project and download one of the binary bundles. You’ll need to be running snv_94 or newer… if your not, BFU to it now. The binary bundle contains 4 packages, install them all.

Once you’ve got things installed, engage COMSTAR (the stmf service) and enable the iSCSI Target Port Provider. If you have been using the old iSCSI Target (iscsitadm, and co.) disable it first.

root@ultra ~$ svcadm enable system/stmf
root@ultra ~$ svcadm enable iscsi/target
root@ultra ~$ svcs -a | tail
...
online          0:04:29 svc:/system/stmf:default
online          0:04:29 svc:/network/iscsi/target:default

The commands you’ll want to explore are the following:

• itadm: The iSCSI Target admin command which controls the COMSTAR iSCSI Port Provider
• smtfadm: The STMF management CLI; it is the most central of all COMSTAR related commands which maps Port Provider resources (targets) to Logical Unit (LU) Providers (storage)
• sbdadm: The SCSI Block Disk (LU Provider) CLI

Creating iSCSI Targets with COMSTAR is strange at first because unlike the current Solaris implementation (“iscsitadm”) you do not specify what storage to use when you create an iSCSI target. To review, with the current target you would use a command like this:

root@ultra ~$ iscsitadm create target --size 10g --backing-storage /dev/rdsk/c5t0d0s0

Because of COMSTAR’s abstraction through STMF, iSCSI Targets are now independant of the storage behind them; thus we create iSCSI targets using the itadm command like so:

root@ultra ~$ itadm create-target
Target iqn.1986-03.com.sun:02:98110339-45ff-ce0c-9e28-e6a330eddd92 successfully created

root@ultra ~$ itadm list-target -v
TARGET NAME                                                  STATE    SESSIONS
iqn.1986-03.com.sun:02:98110339-45ff-ce0c-9e28-e6a330eddd92  online   0
        alias:                  -
        auth:                   none
        targetchapuser:         -
        targetchapsecret:       unset
        tpg-tags:               default

What this iSCSI target will point to is handled by STMF, which will map this target to an actual resource. So the next step is to create the actual Logical Unit resource using “sbdadm” for SCSI block disk. I’ll create a ZFS Volume for this purpose:

root@ultra ~$ zfs create pool/comstar
root@ultra ~$ zfs create -V 10g pool/comstar/vol_01


root@ultra ~$ sbdadm create-lu /dev/zvol/rdsk/pool/comstar/vol_01

Created the following LU:

              GUID                    DATA SIZE           SOURCE
--------------------------------  -------------------  ----------------
600144f077000000000048c62bf00001      10737352704      /dev/zvol/rdsk/pool/comstar/vol_01

So now we have our Logical Unit (LU) to plug into our target. We now need to use “stmfadm” to map them together into a “view”. Storage administrators will no doubt be familiar with the task of mapping Volumes or Disks to Targets, this is the same task. Views are linked to a LU, thus a view can only refer to a single LU, but a single LU can have multiple views (more about this later).

Views or Mapping, comes in two forms: Simple and Selective. In simple mode you allow anything to map to the LU… in selective you actually create “host groups” (which initiators/clients can see the LU), target groups (which targets, or port providers, will be able to provide access to the LU), etc. To keep our tutorial here short I’ll opt for the “Simple Mapping” method.

root@ultra ~$ stmfadm list-lu -v
LU Name: 600144F077000000000048C62BF00001
    Operational Status: Online
    Provider Name     : sbd
    Alias             : /dev/zvol/rdsk/pool/comstar/vol_01
    View Entry Count  : 0
root@ultra ~$ stmfadm list-target -v
Target: iqn.1986-03.com.sun:02:98110339-45ff-ce0c-9e28-e6a330eddd92
    Operational Status: Online
    Provider Name     : iscsit
    Alias             : -
    Sessions          : 0


root@ultra ~$ stmfadm list-view -l 600144F077000000000048C62BF00001
stmfadm: 600144f077000000000048c62bf00001: no views found

Here I’ve simply used “stmfadm” to list my LU’s and Targets. Creating the view/mapping is simple:

root@ultra ~$ stmfadm add-view 600144F077000000000048C62BF00001
root@ultra ~$ stmfadm list-view --lu-name 600144F077000000000048C62BF00001
View Entry: 0
    Host group   : All
    Target group : All
    LUN          : 0

Presto chango! Its done! Seriously. Its that easy. Here is a look from my MacBook Pro running OS X and the amazing globalSAN iSCSI Initiator for OS X:

In full disclosure, while the device is available for some reason I can’t partition it… but I’ll work that out shortly.

COMSTAR is an amazing thing and so far the iSCSI Target looks good. iSNS support, TPG support, RADIUS/CHAP support, and of course iSER support are already there and humming along and I can’t wait to test these, especially iSNS integration.

Please note, this is heading toward but is not yet integrated. ZFS “shareiscsi=on” support is not currently available but will when the time comes.

A very warm thank you to the team who’s been working on this.