Compressed Secure ZFS Archives

ZFS is an amazing technology. Its snapshot capability quickly changes the way you work on the desktop or server. Add replication via zfs send .. and zfs recv .. and life gets even better.

Jeff, Bill, and crew, geniuses that they are, did something very special and exceptionally powerful that you might be overlooking… ZFS replication is just a simple stream. Simply brilliant that is, because that means you can dump a ZFS Snapshot into a single file which we would commonly call an “archive”. Example:

root@aeon ~$ zfs snapshot local@mysnapshot
root@aeon ~$ zfs send local@mysnapshot > /export/local-snapshot.zarchive

You can recover that “archive” by piping it into zfs recv….

Now, consider the possibilities if you will… we have the ability to pipe this output from one tool to the next. So what if we take a snapshot, then we zfs send… that snapshot through bzip2 to compress it, and then into “encrypt” to encrypt it and output that to an “archive” file?

root@ultra ~$ zfs create pool/test
root@ultra ~$ cp -r /var/adm/* /pool/test/

root@ultra ~$ zfs list pool/test
NAME        USED  AVAIL  REFER  MOUNTPOINT
pool/test   208K   350G   208K  /pool/test

root@ultra ~$ zfs snapshot pool/test@snapshot-`date +%m-%d-%y`

root@ultra ~$ zfs send pool/test@snapshot-`date +%m-%d-%y` | bzip2 --stdout | encrypt -a aes -o /tmp/
pool.test.snapshot-`date +%m-%d-%y`.archive
Enter key:   <---  "zfs rules"

root@ultra ~$ zfs get refer pool/test@snapshot-`date +%m-%d-%y`
NAME                         PROPERTY    VALUE                        SOURCE
pool/test@snapshot-10-24-07  referenced  208K                         -
root@ultra ~$ ls -alh /tmp/pool.test.snapshot-10-24-07.archive
-rw-r--r--   1 root     root         11K Oct 24 13:17 /tmp/pool.test.snapshot-10-24-07.archive

Now we just turn that around to restore it...

root@ultra ~$ zfs destroy -r pool/test

root@ultra ~$ decrypt -a aes -i /tmp/pool.test.snapshot-10-24-07.archive | bunzip2 --stdout |  zfs recv -d pool
Enter key:
root@ultra ~$ zfs list pool/test
NAME        USED  AVAIL  REFER  MOUNTPOINT
pool/test   208K   350G   208K  /pool/test

Is that kool or what? Power, lots of power... with ZFS Snapshots you can replicate and pipe your way to glory!

14 Responses to “Compressed Secure ZFS Archives”

  1. Phil Davis says:

    You should mention that zfs send man page specifically says:

    “The format of the stream is evolving. No backwards compatibility is guaranteed. You may not be able to receive your streams on future versions of ZFS.”

    which is a big problem for an archive format!

  2. benr says:

    So at the worse, you’ll need to put on the same OS that you made the archive with… this is commonly the case with other forms of archives sadly, they just bury it in the fine print.

    Priority #1 is that you HAVE the data.

  3. Gavin Sandie says:

    Man this sounds great!

    I’ve recently started properly exploring solaris and zfs and am loving the technology. It might even replace my debian desktop as long as I can get E up and running :)

  4. Wes Felter says:

    Why stop there? How about piping it into a tape library?

  5. benr says:

    Wes: Exactly! :)

  6. Alopro says:

    Business opportunity for self motivated people that can work from home
    marketing a range of Aloe Vera
    Products. Earn unlimited funds by developing your own sales distributors in
    any World markets.

    Please visit us.

    http://www.mlm-mall.com/

  7. John Doe says:

    fcd0b4bb28545932b6e46c7899618c28

  8. I have never got an idea that such archives are being introduced which are going to be very helpful and useful too. Thanks for sharing this information. I m sure your blog will continue to update us on such software’s.

  9. Your post was very nice……………

  10. ZFS now has built-in deduplication. If you already know what dedup is and why you want it, you can skip the next couple of sections.

  11. excellent article , I added you to my http://www.china-wholesale-directory.com Top China Wholesalers category.. thanks for sharing the article!

  12. ugg sale says:

    Very interesting & professional site. You done great work.