Let there be great rejoicing! Oracle 11g R2 has been released for X86!!! So snubbed was Solaris/X86 that there was no release for Oracle 11g (R1)… but that time has passed and R2 is available on BOTH SPARC & X86. w00t! Only 5 years of checking daily… I knew it would pay off eventually. 
Just an update on the acquisition front… Oracle gets more time to respond to EC antitrust concerns. “The deadline for a final ruling has been put back to Jan. 27 from Jan. 19, which amounts to six additional working days for Oracle to win over the skeptical regulator.”
It’s become crystal clear, for those not following the issue, that this is really all about MySQL. As I and others have sited repeatedly, the de facto standard MySQL engine for enterprise deployments is InnoDB which is already owned by Oracle, which really puts a big dent in the argument. All this makes you wonder, would Oracle have still acquired Sun if they didn’t own MySQL? I tend to think, yes. Which makes that deal seem all the more ridiculous. All the same, Sun paid $1B for it, so the suggestion that Oracle should just let that entity break back off is even more ridiculous, not to mention just bad business.
Nevada Build 128b (snv_128) is now closed and available as BFU or source tarball. This means that those who want to play with ZFS Dedup but don’t want to build from source can give it a go.
It should be said that there have been a lot of exciting enhancements to Nevada over the last couple of builds. Here are some of the changes in the last couple builds:
If your not running at least Build 121 your really behind the times, and I highly recommend that if you have the time to install SX:CE 127 and BFU up to 128… or, if your busy with the holidays, make sure you set aside some time in December to really dig into the new hotness when SX:CE 128 releases.
UPDATE: It was announced that there shall be No SXCE 128 Release due to CR 6903705. So if you were waiting to avoid a BFU or full compile you should bother waiting unless you can hold out till mid December for 129.
Also, do remember, the clock is ticking on SX:CE. I still have hopes for a reprieve from the powers that be to keep SX:CE alive, but you never know. If that happens either be prepared to get really comfortable with BFU or code building, or get ready to embrace Indiana with both arms. Solaris 10 proper is of course an option, but not a great one.
UPDATE 2: Changed link to the re-spin snv_128b.
Role models are something we have few of; sad that perhaps the most recent one comes from a beer commercial:
I mean, come on… his advice on careers “Find what you don’t do well…. and don’t do that thing.” Classic!
Need something more expansive? Learn Chinese! If you find it difficult, try to learn Japanesse… and then you’ll go back and appreciate how much easier Chinese languages are.
Not intellectual enough? Need to stretch those brain cells a bit more? Then, I ask, what is justice? As a Christian I have all those answers, laid down thousands of years ago, but since apparently folks like to re-invent the wheel (something King Solomon explained to us about 1,000 BC… “There is nothing new under the sun”), try Harvard’s Michael Sandel discussion on Justice. A fun and engaging discussion in one of Harvard’s beautiful facilities, exploring the “Moral Side of Murder”. It’s an enjoyable mental exercise and well expressed.
And finally, since I mentioned Christianity, if you are not a Christian but curious about it, here is a recent and awesome sermon from Mars Hill Church in Seattle. Watch it, argue with it, think about it, I think you’ll enjoy it. Pastor Mark is always fun.
If your reading this post on an aggregator or via RSS and don’t see the embedded video, just come here to cuddletech to see it properly.
Lots of folks have switched to Mac, its the most commonly used laptop in the Bay Area now. Sometimes people give me flack for using it, but I’ll tell you why I use a Mac laptop:
Now, please note that I do not have nor do I ever plan to have a Mac desktop! For my daily work I need a real UNIX Workstation. I prefer to work with Enlightenment, Eterm, and have a real Solaris system on which to work. Without my desktop I can’t accomplish real work, but for the road I need my MacBook Pro.
So here are some of my “must have apps” for OS X:
I’ll add some more to the honorable mention list…
On the hardware side, every UNIX Admin must be able to access an RS-232 serial console. This fact kept me away from Mac laptops for a long time. Which is why you need this:
The Keyspan Serial-USB Adapter. Buy one, download the Keyspan Assistant software and install Zterm. Good to go!
Finally let me point out 2 things which are already in Leopard that you may not be aware of:
First, with the OS on the Install disk is the Apple Xcode IDE. Along with Xcode is the koolest GUI for DTrace you’ll ever see: Instruments Its really amazingly awesome and a must see.
Secondly, OS X includes native Kerberos support and a ticket management GUI which is sort of buried: /System/Library/CoreServices/Kerberos. If you use Kerberos at all drag that binary onto your doc for quick access. Several other hidden gems can be found in the same directory.
I’m really pleased to announce that Intel Capital has invested in Joyent.
This is a really exciting thing for us. This is the first time we’ve taken funding. We’ve really been proud of the fact that we haven’t needed funding, but the benefits that come along with an investment from Intel are fantastic and just that relationship alone is exciting.
This is a big announcement not only for Joyent, but for OpenSolaris as well. We’re thrilled that Intel supports not only what we’re doing, but also how we’re doing it. Combined with our recent expansion into China, we have a lot to be happy about.
My talk at LISA is now available. This is a 1 hour version of the ZFS in the Trenches talk. As always I hope that you find it informative and at least a little entertaining. Slides are here).
I also want to take this opportunity to say a heart felt thank you to Deirdré Straughan, Lynn Rohrer, and Teresa Giacomini.
Because of Deirdre countless people around the globe can participate and learn from important events. Not only does she spend a mind-boggling amount of time going to these events, but she has done a fantastic job producing very high quality content, and I think is setting the bar in community video presentation. We just don’t get this kind of content from other top tier vendors and I really hope they take notice of her efforts and the benefit to Sun’s current and prospective customer bases.
So please join me in extending your support and appreciation to Deirdre and everyone at Sun that makes these events accessible to the whole world!
For sometime now I’ve gone back and forth on what is my personally preferred (LDAP) directory server; in particular between Sun Directory Server Enterprise Edition, OpenDS, and OpenLDAP. Each has advantages and trade-offs:
So I put it to my loyal and educated readers… which is your directory of choice?
ZFS Deduplication was recently putback (Sun terminology for “commit”) to ON (Solaris’s primary codebase). That means it should go out at snv_128 (Build 128) due later this week.
Unable to wait for the BFU archives I resorted to actually building the code myself to play; something I’ve not felt the burning need to do for at least 2 years (I’ll blog about that shortly). Here’s the initial review…
In typical fashion putting ZFS Dedup to work is a trivial task. Zpools are created in the normal way, the dedup feature is enabled on a per-dataset basis and therefore is a simple matter of turning it on:
root@quadra ~$ zpool create stick c4t0d0 root@quadra ~$ zpool get all stick NAME PROPERTY VALUE SOURCE stick size 3.75G - stick capacity 0% - stick altroot - default stick health ONLINE - stick guid 12142487970365036186 default stick version 21 default stick bootfs - default stick delegation on default stick autoreplace off default stick cachefile - default stick failmode wait default stick listsnapshots off default stick autoexpand off default stick dedupratio 1.00x - stick free 3.75G - stick allocated 76.5K -
Notice that there is no option to enable dedup for the pool, however there is a read-only “dedupratio” key. Because ZFS properties are inherited by child datasets we’ll enable dedup on the root dataset, in this case “stick”:
root@quadra ~$ zfs set dedup=on stick
Done! That’s it. Really, you’re done! Stop reading this now. 
… ok, maybe I’ll go into it a bit more.
As with many ZFS Dataset Properties, there can be more than one setting. The default value of the “dedup” properties is “off”. It can also be set to “on”, “sha256″, “verify”, or “fletcher4,verify”. “on” is simply a pseudonym for “sha256″. “verify” is a pseudonym for “sha256,verify” and enables an ability to detect and correct hash collisions, however this is very system intensive and is not recommended for casual use, if you require absolute integrity at all costs, go for it, but test your workload first. Phrases like “hash collision” can cause a panic, but remember that the odds are astronomical. For details on this see Jeff Bonwick’s post on ZFS Dedup.
So, now for some testing. I’ve created my “stick” pool on a new 4GB micro-USB stick and enabled dedup. Lets copy in a bunch of JPEG’s to several directories and see what happens:
root@quadra ~$ zfs list stick NAME USED AVAIL REFER MOUNTPOINT stick 73.5K 3.69G 21K /stick root@quadra ~$ mkdir /stick/userA root@quadra ~$ mkdir /stick/userB root@quadra ~$ mkdir /stick/userC root@quadra ~$ cd img root@quadra img$ time cp * /stick/userA real 0m15.395s user 0m0.005s sys 0m0.174s root@quadra img$ time cp * /stick/userB real 0m15.952s user 0m0.004s sys 0m0.112s root@quadra img$ time cp * /stick/userC real 0m2.347s user 0m0.004s sys 0m0.125s root@quadra img$ zfs list stick NAME USED AVAIL REFER MOUNTPOINT stick 203M 3.62G 203M /stick root@quadra img$ cd /stick/userA/ root@quadra userA$ du -sh . 74M .
OK, notice that I’m copying in 74MB of data, 3 times, each to a different directory. (Its slow because its a crappy USB stick.) If we run du it registers the proper size, if we look at zfs list it shows the full size of 203MB. In fact, if I look at the dataset properties I have no indication at all of its on-disk size:
root@quadra userA$ zfs get all stick NAME PROPERTY VALUE SOURCE stick type filesystem - stick creation Tue Nov 10 0:07 2009 - stick used 220M - stick available 3.62G - stick referenced 220M - stick compressratio 1.00x - stick mounted yes - stick quota none default stick reservation none default ...
So here’s the magic… look at the pool size:
root@quadra ~$ zpool list stick NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT stick 3.75G 72.5M 3.68G 1% 3.06x ONLINE -
Beautiful. 72.5MB allocated and we correctly see the dedup ratio of 3 (less than the file sizes, leading me to believe there are some duplicate images, which I don’t doubt).
Yet again, ZFS makes it “just work”. And you don’t need a big huge expensive peice of gear, I’m deduping on this:
Suck it Data Domain.
For the elite ZFS Internals hackers out there, you can get a closer look at dedup using zdb -S (thanks to Jeff Victor for the tip):
root@quadra ~$ zdb -S stick
Simulated DDT histogram:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 623 69.9M 69.9M 69.9M 1.83K 210M 210M 210M
4 14 1.63M 1.63M 1.63M 84 9.8M 9.8M 9.8M
Total 637 71.5M 71.5M 71.5M 1.91K 219M 219M 219M
dedup = 3.07, compress = 1.00, copies = 1.00, dedup * compress / copies = 3.07
So now its time to really beat on this thing and see if and where it breaks. Dedup for the masses is coming in the mail!!!
benr@quadra ~$ zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT quadra 928G 518G 410G 55% 1.00x ONLINE -
Shiny.
More to come soon….
You are currently browsing the Cuddletech blog archives for November, 2009.
