In this lesson we're going to build a Simple RAID. We're
going to create it from our 4 9G drives which we added earlier.
We're going to build this volume object by object, and we're going
to put the objects together until we've built out volume. If
it sounds like I'm talking about Lego's, it's not too far from
the truth. It's very similar.
First, we need to great SubDisks from our VM Disks. We'll
need these SubDisks to build a plex, later. As always, I'm
going to start by verifying my current configuration with "vxprint"
and then I'm going to create my subdisks. This creation will
be done with one-line per subdisk creation. To do this creation
I'm going to use, our hero, VxMake. The syntax for building
a SubDisk from a VM Disk using VxMake is this:
vxmake sd <subdiskname> <VMDiskName>,<offset>,<len> |
The syntax is pretty easy. Vxmake is the command, "sd"
specifies that a subdisk is being created. "subdiskname" is the
name that the new subdisk will be know as; typically this name
is the name of the VM Disk is was created from (for reference)
with a "-01" after it. The "-0X" would be number sequentially
for each new subdisk created from that VM Disk. The "vmdiskname"
is the name of the VM Disk we want to create this subdisk from.
"offset" would be used for, um, setting offset. We'll talk more
about this later; generally you won't need it. "len" is the length
of the disk in sectors. To allocate the whole VM Disk to the new
subdisk you could just "cut-and-paste" the length from the vxprint
output. That's it! That's all you need to know and do to create
a subdisk. Let me interject now that creating subdisks is the
Veritas way to "partition" disks. For instance, if you wanted to
create 2 volumes, striped across 4 disks, and you only had 4 disks
you could make 2 subdisks from each VM Disk and then use the "disk0?-01"
subdisks for the first volume, and the "disk0?-02" subdisks for the second.
I hope this helps clarify subdisks.
Now, let's see what it looks like to create our subdisks, I'm
going to create 4 subdisks one after the other, after checking vxprint.
Here it is:
# vxprint
Disk group: rootdg
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg rootdg rootdg - - - - - -
dm disk01 c2t0d0s2 - 17678493 - - - -
dm disk02 c2t1d0s2 - 17678493 - - - -
dm disk03 c2t2d0s2 - 17678493 - - - -
dm disk04 c2t6d0s2 - 17678493 - - - -
# vxmake sd disk01-01 disk01,0,17678493
# vxmake sd disk02-01 disk02,0,17678493
# vxmake sd disk03-01 disk03,0,17678493
# vxmake sd disk04-01 disk04,0,17678493
# |
That's it! We've now made our subdisks! Notice the "len" is identical
to the "Length" from the vxprint output. Also notice the naming conventions.
Very simple huh? Further, I want to point out that vxmake doesn't output
anything in return. If the operation was successful it will return
a prompt. If there is an error you'll get the error message. When creating
several (more than 6) subdisks at one time, make sure to check vxprint
often in case you make a typo that is "syntactically correct". With that
said, lets look at what vxprint outputs now:
# vxprint
Disk group: rootdg
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg rootdg rootdg - - - - - -
dm disk01 c2t0d0s2 - 17678493 - - - -
dm disk02 c2t1d0s2 - 17678493 - - - -
dm disk03 c2t2d0s2 - 17678493 - - - -
dm disk04 c2t6d0s2 - 17678493 - - - -
sd disk01-01 - ENABLED 17678493 - - - -
sd disk02-01 - ENABLED 17678493 - - - -
sd disk03-01 - ENABLED 17678493 - - - -
sd disk04-01 - ENABLED 17678493 - - - - |
See? We're just building and building. Next, we need to build our
plex. The plex is actually the trickiest part, because striping and RAID5
is done at the plex level, not the volume level (remember that the volume
is just a dumb container for plexes.... plexes do the work). Because
we're going to build a "simple" plex we don't need to give any fancy options.
In this case, the syntax will be:
Vxmake is the command we're using. "plex" tells vxmake that
we want to build a new plex. And "plexname" is the name of our plex.
You can name the plex anything you want. However, normally 95% of
people name plexes as the volume name, dash 01, after the volume
they will be used in. This is completely voluntary and is merely
helpful to increase readability when troubleshooting. In our example
I'm going to name our plex "dataplex1". Here's what it looked
like when I build the plex, followed by the vxprint output:
# vxmake plex dataplex1
# vxprint
Disk group: rootdg
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg rootdg rootdg - - - - - -
dm disk01 c2t0d0s2 - 17678493 - - - -
dm disk02 c2t1d0s2 - 17678493 - - - -
dm disk03 c2t2d0s2 - 17678493 - - - -
dm disk04 c2t6d0s2 - 17678493 - - - -
sd disk01-01 - ENABLED 17678493 - - - -
sd disk02-01 - ENABLED 17678493 - - - -
sd disk03-01 - ENABLED 17678493 - - - -
sd disk04-01 - ENABLED 17678493 - - - -
pl dataplex1 - DISABLED 0 - - - -
# |
See the new plex in the vxprint output? Notice that there are no
subdisks in it. With vxmake I can actually specify which subdisks should
be assigned to the plex, but I want you to do this the long way. This
is 100% manual!
To add the subdisks to our new plex, we're going to use
the "vxsd" command, which is the subdisk control tool. Vxsd has
all kinds of options and functions, but we're going to use it
for "subdisk association". As the name suggests, we're merely
means we're "associating" the subdisks, which implies we can later
"disassociate" the subdisks. Get the idea? We'll look at this more
later. The syntax used to associate disk using vxsd is:
vxsd assoc <plexname> <subdisk> <subdisk> ..... |
"vxsd" is the command. "assoc" means we're associating
disks to a plex. "plexname" is the name of the plex to associate
the subdisks with. And then the rest of the command is the
list of subdisks we're associating, separated by spaces. I'm going
to be associating all 4 of our disks, and then check my change
with vxprint. Here's what it looks like:
# vxsd assoc dataplex1 disk01-01 disk02-01 disk03-01 disk04-01
# vxprint
Disk group: rootdg
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg rootdg rootdg - - - - - -
dm disk01 c2t0d0s2 - 17678493 - - - -
dm disk02 c2t1d0s2 - 17678493 - - - -
dm disk03 c2t2d0s2 - 17678493 - - - -
dm disk04 c2t6d0s2 - 17678493 - - - -
pl dataplex1 - DISABLED 70713972 - - - -
sd disk01-01 dataplex1 ENABLED 17678493 0 - - -
sd disk02-01 dataplex1 ENABLED 17678493 17678493 - - -
sd disk03-01 dataplex1 ENABLED 17678493 35356986 - - -
sd disk04-01 dataplex1 ENABLED 17678493 53035479 - - -
# |
Looking good so far! Now that we're got a plex built, we only
need to attach the now completed plex to a volume. We'll build a
volume using vxmake, and the following syntax:
vxmake -U <usagetype> vol <volname> plex=<plexname> |
This one is alittle more confusing. "vxmake" is the
command we're using. Here we'll use an argument "-U" after
which we specify the "usage type". We won't talk about
usage types now, but for a volume to contain a file system
you'll need to use the "fsgen" usage type. "vol" tells
vxmake we're building a new volume. "volname" is the
name we want our new volume to have. And this time,
we are going to let vxmake attach the plex for us, instead
of having to do it ourselves. We do this with "plex="
followed by the name(s) of the plex(es) we want to attach.
We could attach multiple plexes by listing them separated
by coma's (then you'd have a mirror!), but we're only
going to attach our "dataplex1" plex. That's it. So lets
build the volume and then run vxprint. Notice I'm going
to run vxprint this type with the "-hrt" (which I remember
as "dash hurt") which shows us much more detail, than
normal vxprint output does. Let's look at it, and then we'll
discus it:
# vxmake -U fsgen vol datavol plex=dataplex1
# vxprint -hrt
Disk group: rootdg
DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
dg rootdg default default 0 952738334.1025.nexus6
dm disk01 c2t0d0s2 sliced 3590 17678493 -
dm disk02 c2t1d0s2 sliced 3590 17678493 -
dm disk03 c2t2d0s2 sliced 3590 17678493 -
dm disk04 c2t6d0s2 sliced 3590 17678493 -
v datavol fsgen DISABLED EMPTY 70713972 ROUND -
pl dataplex1 datavol DISABLED EMPTY 70713972 CONCAT - RW
sd disk01-01 dataplex1 disk01 0 17678493 0 c2t0d0 ENA
sd disk02-01 dataplex1 disk02 0 17678493 17678493 c2t1d0 ENA
sd disk03-01 dataplex1 disk03 0 17678493 35356986 c2t2d0 ENA
sd disk04-01 dataplex1 disk04 0 17678493 53035479 c2t6d0 ENA
# |
Kool huh? Don't let all the vxprint headers generated because of
the "-hrt" scare you. They are keys for each lines output. The two
letter designations should start looking familiar, with the exception of
the "SV" which is something you'll rarely see, and is for the advanced course.
Anyway, What I wanted you to see is that with the "-hrt" on vxprint we
see on the "pl" (plex) line of the output vxprint reports that "dataplex1"
is a "CONCAT" or concatenated RAID (aka Simple RAID). Also notice that
each time we put objects together that they regroup themselves in the vxprint
output. This just makes everything easier and easier to read and understand
quickly.
We've now got a volume! We now need to start the volume and create
a filesystem on it, after which we can mount it. We can start a volume
with the "vxvol" command (which is used for vx volume control, and also
has tons of kool uses). The syntax is just:
I won't even bore you with the analysis, this is self explanatory.
So let's just do it! We're going to start the volume and then look
at vxprint (YES, AGAIN!!!!):
# vxvol start datavol
# vxprint -hrt
Disk group: rootdg
DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
dg rootdg default default 0 952738334.1025.nexus6
dm disk01 c2t0d0s2 sliced 3590 17678493 -
dm disk02 c2t1d0s2 sliced 3590 17678493 -
dm disk03 c2t2d0s2 sliced 3590 17678493 -
dm disk04 c2t6d0s2 sliced 3590 17678493 -
v datavol fsgen ENABLED ACTIVE 70713972 ROUND -
pl dataplex1 datavol ENABLED ACTIVE 70713972 CONCAT - RW
sd disk01-01 dataplex1 disk01 0 17678493 0 c2t0d0 ENA
sd disk02-01 dataplex1 disk02 0 17678493 17678493 c2t1d0 ENA
sd disk03-01 dataplex1 disk03 0 17678493 35356986 c2t2d0 ENA
sd disk04-01 dataplex1 disk04 0 17678493 53035479 c2t6d0 ENA
# |
Here's the big key to notice. The "KSTATE" (Kernel State) and "STATE"
changed from "DISABLED" and "EMPTY", to "ENABLED" and "ACTIVE". By starting
the volume it's suddenly jumped to life.
Having fun yet? I just love Veritas. Isn't this logical?
If your sailing through this, your doing better than you image. Remember,
this is the "hard" way!
Okey. It's show time. We've got a live volume. We need to
create a filesystem on it, and mount it.
Like many other volume managers, the volumes are accessible
via a device file in the "/dev" file tree. Veritas volumes
can be found in /dev/vx/dsk. Let me show you:
# ls -al /dev/vx/dsk
total 6
drwxr-xr-x 3 root root 512 Mar 29 23:37 .
drwxr-xr-x 6 root other 512 Mar 10 17:32 ..
brw------- 1 root root 67, 5 Mar 29 23:37 datavol
drwxr-xr-x 2 root root 512 Mar 29 23:37 rootdg
# |
Wondering about the "rootdg" showing up? Because our
volume is in the "default" disk group (rootdg) we see it
in the /dev/vx/dsk directory. If we created a new disk group,
and built a volume in it, the volume would be in
/dev/vx/dsk/dgname/volname. Simple enough.
Just like any other volume manager, we can access a volume
just like a disk, and we see that /dev/vx/dsk/datavol is a block
device, so we'll create our filesystem on it. I'm going to
use newfs (which defaults UFS) to create the file system on the
volume:
# newfs /dev/vx/dsk/datavol
newfs: construct a new file system /dev/vx/rdsk/datavol: (y/n)? y
/dev/vx/rdsk/datavol: 70713972 sectors in 34529 cylinders of 32 tracks, 64 sectors
34528.3MB in 705 cyl groups (49 c/g, 49.00MB/g, 6144 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 100448, 200864, 301280, 401696, 502112, 602528, 702944, 803360, 903776,
1004192, 1104608, 1205024, 1305440, 1405856, 1506272, 1606688, 1707104,
1807520, 1907936, 2008352, 2108768, 2209184, 2309600, 2410016, 2510432,
snipped out
# |
Newfs created the filesystem, just like it would if we used
a normal partitioned disk. Notice that the size of this filesystem
is 34.5G, and we used 4 9G disks, so this is about right. Now we're
ready to mount the filesystem, which is just like normal. If we
wanted to mount our new volume to "/vxrocks", we'd use the mount
command like this:
# mount /dev/vx/dsk/datavol /vxrocks
# |
That's it!!!! We're all done! I have a harder time
getting dressed in the morning! Again, this was "the hard way".
Even though you might not always build volumes this way (I do...)
you will at least now have a better understanding of how things
are put together, and how objects work with each other.