Using Volumes as iSCSI Targets

The target implementation used earlier can be used only on block devices at the present. This means that we can create block devices and use those, allowing us to use Linux RAID, LVM, Vinum, EVMS or any other volume manager in order to create more efficient storage solutions. In the following procedure we'll use the Linux RAID tools to create a 4 disk RAID0 (striped) volume and then make it avalible as an iSCSI target.


I am personally not a fan of the Linux RAID tools. I use them here simply because it's quick and easy, and so that this procedure is about iSCSI and not an LVM or EVMS tutorial. I highly suggest that if you plan to seriuosly use volumes for iSCSI that you consider using LVM, Vinum, or EVMS.

Procedure 4. Setting up RAID Targets

  1. Ensure that Linux RAID (and LVM if you want it) are compiled into your kernel, and that the mkraid tool is installed.

  2. Create a raidtab for your volume, such as the following.

    [root@nexus /etc]# cat raidtab 
    raiddev /dev/md0
            raid-level      0
            nr-raid-disks   4
            persistent-superblock 1
            chunk-size     8
            device          /dev/sdd1
            raid-disk       0
            device          /dev/sde1
            raid-disk       1
            device          /dev/sdf1
            raid-disk       2
            device          /dev/sdg1
            raid-disk       3
    [root@nexus /etc]# 
  3. Create the volume.

    [root@nexus /etc]# mkraid /dev/md0
    handling MD device /dev/md0
    analyzing super-block
    disk 0: /dev/sdd1, 17782768kB, raid superblock at 17782656kB
    disk 1: /dev/sde1, 17782768kB, raid superblock at 17782656kB
    disk 2: /dev/sdf1, 17782768kB, raid superblock at 17782656kB
    disk 3: /dev/sdg1, 17782768kB, raid superblock at 17782656kB
    [root@nexus /etc]# 
  4. Add the metadevice to your /etc/iscsid.conf.

    # 4 Disk Linux RAID Stripe
            Lun 0 /dev/md0
            Alias RAID0
  5. If iSCSI is in use, stop both the initiator and the target daemons now.

    [root@nexus /etc]# /etc/init.d/iscsi stop
    Stopping iSCSI: sync umount sync iscsid iscsi
    [root@nexus /etc]# /etc/init.d/iscsid  stop
    Stopping iSCSI target.
  6. Start the target daemon and then the initiator. Watch syslog while you do this to make sure your new target is auto-discovered.

    [root@nexus /etc]# /etc/init.d/iscsid start
    Starting iSCSI target.
    [root@nexus /etc]# /etc/init.d/iscsi start
    Starting iSCSI: iscsi iscsid fsck/mount

    In syslog (/var/log/messages) we see a new target auto-discovered.

    kernel: scsi singledevice 1 0 2 0
    kernel:   Vendor: LINUX     Model: ISCSI             Rev: 0   
    kernel:   Type:   Direct-Access                      ANSI SCSI revision: 03
    kernel: Attached scsi disk sdj at scsi1, channel 0, id 2, lun 0
    kernel: SCSI device sdj: 142261248 512-byte hdwr sectors (72838 MB)
    kernel:  sdj: unknown partition table
  7. Now partition, label and create a filesystem on the new target.

    [root@nexus /etc]# fdisk /dev/sdj
    	( Removed for Clarity )
    Command (m for help): p
    Disk /dev/sdj: 64 heads, 32 sectors, 69463 cylinders
    Units = cylinders of 2048 * 512 bytes
       Device Boot    Start       End    Blocks   Id  System
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    Partition number (1-4): 1
    First cylinder (1-69463, default 1): 
    Using default value 1
    Last cylinder or +size or +sizeM or +sizeK (1-69463, default 69463): 
    Using default value 69463
    Command (m for help): p
    Disk /dev/sdj: 64 heads, 32 sectors, 69463 cylinders
    Units = cylinders of 2048 * 512 bytes
       Device Boot    Start       End    Blocks   Id  System
    /dev/sdj1             1     69463  71130096   83  Linux
    Command (m for help): w
    The partition table has been altered!
    Calling ioctl() to re-read partition table.
    	( Removed for Clarity )
    Syncing disks.
    [root@nexus /etc]# mkfs -t jfs /dev/sdj1
    mkfs.jfs version 1.1.0, 20-Nov-2002
    Warning!  All data on device /dev/sdj1 will be lost!
    Continue? (Y/N) y
    Format completed successfully.
    71130096 kilobytes total disk space.
  8. Now mount it.

    [root@nexus /]# mkdir /iscsi_raid
    [root@nexus /]# mount -t jfs /dev/sdj1 /iscsi_raid 
    [root@nexus /]# df -h
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/hda1             4.8G  4.3G  286M  94% /
    /dev/hda5              13G   11G  2.0G  85% /home
    /dev/sdj1              68G  8.7M   67G   1% /iscsi_raid
  9. Done! Put some data on the new device and watch all the lights blink while you do writes, run some benchmarks, or just watch iostat data while you copy data onto the device.

    [root@nexus benr]# iostat -kt
    Linux 2.4.22 (      04/13/2004
    	( Removed for Clarity )
    avg-cpu:  %user   %nice    %sys %iowait   %idle
               7.00    0.00   50.00    0.00   43.00
    Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
    dev3-0          205.00      8488.00      2248.00       8488       2248
    dev8-0            0.00         0.00         0.00          0          0
    dev8-1            0.00         0.00         0.00          0          0
    dev8-2            0.00         0.00         0.00          0          0
    dev8-3          122.00         0.00      3604.00          0       3604
    dev8-4          120.00         0.00      3596.00          0       3596
    dev8-5          121.00         0.00      3604.00          0       3604
    dev8-6          122.00         0.00      3600.00          0       3600
    dev8-7            0.00         0.00         0.00          0          0
    dev8-8            0.00         0.00         0.00          0          0
    dev8-9          126.00         0.00     14404.00          0      14404