In this lesson we're going to build a Simple RAID. We're going to create it from our 4 9G drives which we added earlier. We're going to build this volume object by object, and we're going to put the objects together until we've built out volume. If it sounds like I'm talking about Lego's, it's not too far from the truth. It's very similar.
First, we need to great SubDisks from our VM Disks. We'll need these SubDisks to build a plex, later. As always, I'm going to start by verifying my current configuration with "vxprint" and then I'm going to create my subdisks. This creation will be done with one-line per subdisk creation. To do this creation I'm going to use, our hero, VxMake. The syntax for building a SubDisk from a VM Disk using VxMake is this:
vxmake sd <subdiskname> <VMDiskName>,<offset>,<len> |
The syntax is pretty easy. Vxmake is the command, "sd" specifies that a subdisk is being created. "subdiskname" is the name that the new subdisk will be know as; typically this name is the name of the VM Disk is was created from (for reference) with a "-01" after it. The "-0X" would be number sequentially for each new subdisk created from that VM Disk. The "vmdiskname" is the name of the VM Disk we want to create this subdisk from. "offset" would be used for, um, setting offset. We'll talk more about this later; generally you won't need it. "len" is the length of the disk in sectors. To allocate the whole VM Disk to the new subdisk you could just "cut-and-paste" the length from the vxprint output. That's it! That's all you need to know and do to create a subdisk. Let me interject now that creating subdisks is the Veritas way to "partition" disks. For instance, if you wanted to create 2 volumes, striped across 4 disks, and you only had 4 disks you could make 2 subdisks from each VM Disk and then use the "disk0?-01" subdisks for the first volume, and the "disk0?-02" subdisks for the second. I hope this helps clarify subdisks.
Now, let's see what it looks like to create our subdisks, I'm going to create 4 subdisks one after the other, after checking vxprint. Here it is:
# vxprint Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg rootdg rootdg - - - - - - dm disk01 c2t0d0s2 - 17678493 - - - - dm disk02 c2t1d0s2 - 17678493 - - - - dm disk03 c2t2d0s2 - 17678493 - - - - dm disk04 c2t6d0s2 - 17678493 - - - - # vxmake sd disk01-01 disk01,0,17678493 # vxmake sd disk02-01 disk02,0,17678493 # vxmake sd disk03-01 disk03,0,17678493 # vxmake sd disk04-01 disk04,0,17678493 # |
That's it! We've now made our subdisks! Notice the "len" is identical to the "Length" from the vxprint output. Also notice the naming conventions. Very simple huh? Further, I want to point out that vxmake doesn't output anything in return. If the operation was successful it will return a prompt. If there is an error you'll get the error message. When creating several (more than 6) subdisks at one time, make sure to check vxprint often in case you make a typo that is "syntactically correct". With that said, lets look at what vxprint outputs now:
# vxprint Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg rootdg rootdg - - - - - - dm disk01 c2t0d0s2 - 17678493 - - - - dm disk02 c2t1d0s2 - 17678493 - - - - dm disk03 c2t2d0s2 - 17678493 - - - - dm disk04 c2t6d0s2 - 17678493 - - - - sd disk01-01 - ENABLED 17678493 - - - - sd disk02-01 - ENABLED 17678493 - - - - sd disk03-01 - ENABLED 17678493 - - - - sd disk04-01 - ENABLED 17678493 - - - - |
See? We're just building and building. Next, we need to build our plex. The plex is actually the trickiest part, because striping and RAID5 is done at the plex level, not the volume level (remember that the volume is just a dumb container for plexes.... plexes do the work). Because we're going to build a "simple" plex we don't need to give any fancy options. In this case, the syntax will be:
vxmake plex plexname |
Vxmake is the command we're using. "plex" tells vxmake that we want to build a new plex. And "plexname" is the name of our plex. You can name the plex anything you want. However, normally 95% of people name plexes as the volume name, dash 01, after the volume they will be used in. This is completely voluntary and is merely helpful to increase readability when troubleshooting. In our example I'm going to name our plex "dataplex1". Here's what it looked like when I build the plex, followed by the vxprint output:
# vxmake plex dataplex1 # vxprint Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg rootdg rootdg - - - - - - dm disk01 c2t0d0s2 - 17678493 - - - - dm disk02 c2t1d0s2 - 17678493 - - - - dm disk03 c2t2d0s2 - 17678493 - - - - dm disk04 c2t6d0s2 - 17678493 - - - - sd disk01-01 - ENABLED 17678493 - - - - sd disk02-01 - ENABLED 17678493 - - - - sd disk03-01 - ENABLED 17678493 - - - - sd disk04-01 - ENABLED 17678493 - - - - pl dataplex1 - DISABLED 0 - - - - # |
See the new plex in the vxprint output? Notice that there are no subdisks in it. With vxmake I can actually specify which subdisks should be assigned to the plex, but I want you to do this the long way. This is 100% manual!
To add the subdisks to our new plex, we're going to use the "vxsd" command, which is the subdisk control tool. Vxsd has all kinds of options and functions, but we're going to use it for "subdisk association". As the name suggests, we're merely means we're "associating" the subdisks, which implies we can later "disassociate" the subdisks. Get the idea? We'll look at this more later. The syntax used to associate disk using vxsd is:
vxsd assoc <plexname> <subdisk> <subdisk> ..... |
"vxsd" is the command. "assoc" means we're associating disks to a plex. "plexname" is the name of the plex to associate the subdisks with. And then the rest of the command is the list of subdisks we're associating, separated by spaces. I'm going to be associating all 4 of our disks, and then check my change with vxprint. Here's what it looks like:
# vxsd assoc dataplex1 disk01-01 disk02-01 disk03-01 disk04-01 # vxprint Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg rootdg rootdg - - - - - - dm disk01 c2t0d0s2 - 17678493 - - - - dm disk02 c2t1d0s2 - 17678493 - - - - dm disk03 c2t2d0s2 - 17678493 - - - - dm disk04 c2t6d0s2 - 17678493 - - - - pl dataplex1 - DISABLED 70713972 - - - - sd disk01-01 dataplex1 ENABLED 17678493 0 - - - sd disk02-01 dataplex1 ENABLED 17678493 17678493 - - - sd disk03-01 dataplex1 ENABLED 17678493 35356986 - - - sd disk04-01 dataplex1 ENABLED 17678493 53035479 - - - # |
Looking good so far! Now that we're got a plex built, we only need to attach the now completed plex to a volume. We'll build a volume using vxmake, and the following syntax:
vxmake -U <usagetype> vol <volname> plex=<plexname> |
This one is alittle more confusing. "vxmake" is the command we're using. Here we'll use an argument "-U" after which we specify the "usage type". We won't talk about usage types now, but for a volume to contain a file system you'll need to use the "fsgen" usage type. "vol" tells vxmake we're building a new volume. "volname" is the name we want our new volume to have. And this time, we are going to let vxmake attach the plex for us, instead of having to do it ourselves. We do this with "plex=" followed by the name(s) of the plex(es) we want to attach. We could attach multiple plexes by listing them separated by coma's (then you'd have a mirror!), but we're only going to attach our "dataplex1" plex. That's it. So lets build the volume and then run vxprint. Notice I'm going to run vxprint this type with the "-hrt" (which I remember as "dash hurt") which shows us much more detail, than normal vxprint output does. Let's look at it, and then we'll discus it:
# vxmake -U fsgen vol datavol plex=dataplex1 # vxprint -hrt Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE dg rootdg default default 0 952738334.1025.nexus6 dm disk01 c2t0d0s2 sliced 3590 17678493 - dm disk02 c2t1d0s2 sliced 3590 17678493 - dm disk03 c2t2d0s2 sliced 3590 17678493 - dm disk04 c2t6d0s2 sliced 3590 17678493 - v datavol fsgen DISABLED EMPTY 70713972 ROUND - pl dataplex1 datavol DISABLED EMPTY 70713972 CONCAT - RW sd disk01-01 dataplex1 disk01 0 17678493 0 c2t0d0 ENA sd disk02-01 dataplex1 disk02 0 17678493 17678493 c2t1d0 ENA sd disk03-01 dataplex1 disk03 0 17678493 35356986 c2t2d0 ENA sd disk04-01 dataplex1 disk04 0 17678493 53035479 c2t6d0 ENA # |
Kool huh? Don't let all the vxprint headers generated because of the "-hrt" scare you. They are keys for each lines output. The two letter designations should start looking familiar, with the exception of the "SV" which is something you'll rarely see, and is for the advanced course. Anyway, What I wanted you to see is that with the "-hrt" on vxprint we see on the "pl" (plex) line of the output vxprint reports that "dataplex1" is a "CONCAT" or concatenated RAID (aka Simple RAID). Also notice that each time we put objects together that they regroup themselves in the vxprint output. This just makes everything easier and easier to read and understand quickly.
We've now got a volume! We now need to start the volume and create a filesystem on it, after which we can mount it. We can start a volume with the "vxvol" command (which is used for vx volume control, and also has tons of kool uses). The syntax is just:
vxvol start <volname> |
I won't even bore you with the analysis, this is self explanatory. So let's just do it! We're going to start the volume and then look at vxprint (YES, AGAIN!!!!):
# vxvol start datavol # vxprint -hrt Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE dg rootdg default default 0 952738334.1025.nexus6 dm disk01 c2t0d0s2 sliced 3590 17678493 - dm disk02 c2t1d0s2 sliced 3590 17678493 - dm disk03 c2t2d0s2 sliced 3590 17678493 - dm disk04 c2t6d0s2 sliced 3590 17678493 - v datavol fsgen ENABLED ACTIVE 70713972 ROUND - pl dataplex1 datavol ENABLED ACTIVE 70713972 CONCAT - RW sd disk01-01 dataplex1 disk01 0 17678493 0 c2t0d0 ENA sd disk02-01 dataplex1 disk02 0 17678493 17678493 c2t1d0 ENA sd disk03-01 dataplex1 disk03 0 17678493 35356986 c2t2d0 ENA sd disk04-01 dataplex1 disk04 0 17678493 53035479 c2t6d0 ENA # |
Here's the big key to notice. The "KSTATE" (Kernel State) and "STATE" changed from "DISABLED" and "EMPTY", to "ENABLED" and "ACTIVE". By starting the volume it's suddenly jumped to life.
Having fun yet? I just love Veritas. Isn't this logical? If your sailing through this, your doing better than you image. Remember, this is the "hard" way!
Okey. It's show time. We've got a live volume. We need to create a filesystem on it, and mount it.
Like many other volume managers, the volumes are accessible via a device file in the "/dev" file tree. Veritas volumes can be found in /dev/vx/dsk. Let me show you:
# ls -al /dev/vx/dsk total 6 drwxr-xr-x 3 root root 512 Mar 29 23:37 . drwxr-xr-x 6 root other 512 Mar 10 17:32 .. brw------- 1 root root 67, 5 Mar 29 23:37 datavol drwxr-xr-x 2 root root 512 Mar 29 23:37 rootdg # |
Wondering about the "rootdg" showing up? Because our volume is in the "default" disk group (rootdg) we see it in the /dev/vx/dsk directory. If we created a new disk group, and built a volume in it, the volume would be in /dev/vx/dsk/dgname/volname. Simple enough.
Just like any other volume manager, we can access a volume just like a disk, and we see that /dev/vx/dsk/datavol is a block device, so we'll create our filesystem on it. I'm going to use newfs (which defaults UFS) to create the file system on the volume:
# newfs /dev/vx/dsk/datavol
newfs: construct a new file system /dev/vx/rdsk/datavol: (y/n)? y
/dev/vx/rdsk/datavol: 70713972 sectors in 34529 cylinders of 32 tracks, 64 sectors
34528.3MB in 705 cyl groups (49 c/g, 49.00MB/g, 6144 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 100448, 200864, 301280, 401696, 502112, 602528, 702944, 803360, 903776,
1004192, 1104608, 1205024, 1305440, 1405856, 1506272, 1606688, 1707104,
1807520, 1907936, 2008352, 2108768, 2209184, 2309600, 2410016, 2510432,
snipped out
# |
Newfs created the filesystem, just like it would if we used a normal partitioned disk. Notice that the size of this filesystem is 34.5G, and we used 4 9G disks, so this is about right. Now we're ready to mount the filesystem, which is just like normal. If we wanted to mount our new volume to "/vxrocks", we'd use the mount command like this:
# mount /dev/vx/dsk/datavol /vxrocks # |
That's it!!!! We're all done! I have a harder time getting dressed in the morning! Again, this was "the hard way". Even though you might not always build volumes this way (I do...) you will at least now have a better understanding of how things are put together, and how objects work with each other.