Understanding ZFS: Replication, Archive and Backup

06 Nov '08 - 21:12 by benr

As with other features of ZFS, the traditionally complex is made simple and straight forward. This simplification can coax administrators into a false complacency.

In ZFS, backup, archive, migration... any activity that fundamentally involves the movement of data from one system to another, is a replication activity. I propose that the traditional idea of weekly backups is, in fact, just really slow crappy replication. An HA Cluster replicates every 5 seconds, but your website replicates once a week.... its really the same thing, just via different interval and possibly different tools. So understand that when I say "replication" I refer to all forms of data movement, both intra- and inter- system.

ZFS replication is preformed through the use of two simplistic subcommands: zfs send and zfs recv. These are commands that utilize STDIN and STDOUT.... and why? Pipes my friend, pipes. Rather than bake piles of functionality into these commands, Matt Ahrens and the ZFS team opted to instead make them very simple and utilize the traditional UNIX ideology of connecting things together for something even better.

Lets look at a simple intra-system example of replication. Lets say that I have a workstation with an couple internal disks, perhaps a RAIDZ, who knows, and I then attach a USB or Firewire external drive on which I create a pool called "backups". Lets now migrate a simple dataset from my local "data" pool to my external drives "backups" pool:

root@quadra ~$ zfs list -r data
NAME                USED  AVAIL  REFER  MOUNTPOINT
data                222K   218M    19K  /data
data/home           114K   200M    24K  /data/home
data/home/benr       18K   200M    18K  /data/home/benr
data/home/conradr    18K   200M    18K  /data/home/conradr
data/home/glennr     18K   200M    18K  /data/home/glennr
data/home/novar      18K   200M    18K  /data/home/novar
data/home/tamr       18K   200M    18K  /data/home/tamr
root@quadra ~$ zfs list -r backups
NAME      USED  AVAIL  REFER  MOUNTPOINT
backups  67.5K   218M    18K  /backups

root@quadra ~$ zfs snapshot data/home/benr@001
root@quadra ~$ zfs send data/home/benr@001 | zfs recv -d backups

root@quadra ~$ zfs list -r backups
NAME                    USED  AVAIL  REFER  MOUNTPOINT
backups                 191K   218M    19K  /backups
backups/home            106K   218M    18K  /backups/home
backups/home/benr        88K   218M    88K  /backups/home/benr
backups/home/benr@001      0      -    88K  -

Lets step through this together.

Replication is always based on a static point in time, meaning a snapshot. We create a snapshot of the dataset(s) we want to replicate, in this case the snapshot "001" of benr's home directory. Using the zfs send command we can send that snapshot to STDOUT. Using a UNIX Pipe, that STDOUT gets sent to the STDIN of the zfs recv command, which has been told via the -d backups argument that I want to preserve the dataset name and heirarchy under the "backups" dataset. This could just as easily a "backups/data-pool" dataset under which things are created, like so:

root@quadra ~$ zfs destroy -r backups/home
root@quadra ~$ zfs create backups/data-pool
root@quadra ~$ zfs send data/home/benr@001 | zfs recv -d backups/data-pool
root@quadra ~$ zfs list -r backups
NAME                              USED  AVAIL  REFER  MOUNTPOINT
backups                           217K   218M    20K  /backups
backups/data-pool                 125K   218M    19K  /backups/data-pool
backups/data-pool/home            106K   218M    18K  /backups/data-pool/home
backups/data-pool/home/benr        88K   218M    88K  /backups/data-pool/home/benr
backups/data-pool/home/benr@001      0      -    88K  -

What about incremental? I mean, I'll want to freshen the copy right? This is done by created another snapshot, and then telling zfs send to only actually send the difference between the two:

root@quadra ~$ cp -r /etc/security/* /data/home/benr
root@quadra ~$ zfs snapshot data/home/benr@002
root@quadra ~$ zfs list -r data/home/benr
NAME                 USED  AVAIL  REFER  MOUNTPOINT
data/home/benr       379K   199M   355K  /data/home/benr
data/home/benr@001    24K      -    88K  -
data/home/benr@002      0      -   355K  -

root@quadra ~$ zfs send -i data/home/benr@001 data/home/benr@002 | zfs recv -d backups/data-pool
root@quadra ~$ zfs list -r backups/data-pool
NAME                              USED  AVAIL  REFER  MOUNTPOINT
backups/data-pool                 417K   217M    19K  /backups/data-pool
backups/data-pool/home            398K   217M    19K  /backups/data-pool/home
backups/data-pool/home/benr       379K   217M   355K  /backups/data-pool/home/benr
backups/data-pool/home/benr@001    24K      -    88K  -
backups/data-pool/home/benr@002      0      -   355K  -

So here I used ZFS send/recv almost exactly as before, but this time I tell zfs send about another snapshot from which to create an incremental. Notice that the zfs recv command didn't change at all.

But what if I want to send it to another system? Easy, pipe the data through ssh (or rsh, or whatever) like so:

root@quadra ~$ zfs send data/home/benr@002 | ssh root@thumper.cuddletech.com zfs recv -d backups/data-pool

So thats the basics... but what does this mean? Lets get creative!

Firstly, we can write a script that every 30 seconds creates a new snapshot, and then thanks to pre-shared SSH keys can use SSH like above to recv the data elsewhere. Add a little error checking and presto! A really nice, simplistic replication scheme. Even if you have a lot of data change, if you copy it every 30 seconds its unlikely to build up into huge chunks that will take very long to move. When it comes to data that changes frequently, the key is to move early and often!

Now, say we don't need that, simple backups are fine. We can create a script that creates a new snapshot each day at midnight, named the day of the week. When Wed comes around the old "wed" snapshot is removed and a new one created, and then we way create a simple script that zfs send/recv's the Friday snapshot every weekend. Simple to do, plus we have those daily snapshots to fall back on in a pinch, hopefully keeping us from going out to a remote copy.

So we've used pipes in a simple way, to securly transport our datastream from one system to another. Consider other unique possiblities, such as piping zfs send... into gzip before sending across the network!

Or.... say what you really want is a portable dump of your ZFS dataset(s). Remember that we're outputting a datastream from zfs send... just re-direct STDOUT to a file!

root@quadra ~$ zfs send data/home/benr@002 > /tmp/home-benr.zdump  
root@quadra ~$ ls -lh /tmp/home-benr.zdump
-rw-r--r-- 1 root root 421K Nov  6 15:14 /tmp/home-benr.zdump

Now lets test a restore from this "zdump":

root@quadra ~$ zfs create backups/dump-restore               
root@quadra ~$ cat /tmp/home-benr.zdump | zfs recv -d backups/dump-restore
root@quadra ~$ zfs list -r backups/dump-restore
NAME                                 USED  AVAIL  REFER  MOUNTPOINT
backups/dump-restore                 392K   217M    19K  /backups/dump-restore
backups/dump-restore/home            373K   217M    18K  /backups/dump-restore/home
backups/dump-restore/home/benr       355K   217M   355K  /backups/dump-restore/home/benr
backups/dump-restore/home/benr@002      0      -   355K  -

Works like a charm! Again, we can use pipes for fun here too. Lets say that we really want a dump that is encrypted and compressed!

root@quadra ~$ pktool genkey keystore=file outkey=zdump.key keytype=aes keylen=128
root@quadra ~$ zfs send data/home/benr@002 | gzip | encrypt -a aes -k zdump.key > /tmp/home_benr-AES256GZ.zdump

So we've output a datastream based on a snapshot (002), compressed it, encrypted it with 128bit AES and then dumped to file. We could just as easily dump it to a tape (/dev/rmt/0cbn or something) for archiving purposes.

Finally, what if we want to work on more than just a single snapshot. What if we want to send all the "home" datasets? For some time now (although just now arriving in Solaris 10) we've had recursive flags for both zfs snapshot and zfs send. Lets give it a try:

root@quadra ~$ zfs snapshot -r data/home@nov6
root@quadra ~$ zfs list -r data/home
NAME                     USED  AVAIL  REFER  MOUNTPOINT
data/home                755K   199M    24K  /data/home
data/home@nov6              0      -    24K  -
data/home/benr           379K   199M   355K  /data/home/benr
data/home/benr@001        24K      -    88K  -
data/home/benr@002          0      -   355K  -
data/home/benr@nov6         0      -   355K  -
data/home/conradr         88K   199M    88K  /data/home/conradr
data/home/conradr@nov6      0      -    88K  -
data/home/glennr          88K   199M    88K  /data/home/glennr
data/home/glennr@nov6       0      -    88K  -
data/home/novar           88K   199M    88K  /data/home/novar
data/home/novar@nov6        0      -    88K  -
data/home/tamr            88K   199M    88K  /data/home/tamr
data/home/tamr@nov6         0      -    88K  -

root@quadra ~$ zfs destroy -r backups/home
root@quadra ~$ zfs list -r backups
NAME      USED  AVAIL  REFER  MOUNTPOINT
backups    86K   218M    20K  /backups

root@quadra ~$ zfs send -R data/home@nov6 | zfs recv -d backups

root@quadra ~$ zfs list -r backups
NAME                        USED  AVAIL  REFER  MOUNTPOINT
backups                     902K   217M    18K  /backups
backups/home                755K   199M    24K  /backups/home
backups/home@nov6              0      -    24K  -
backups/home/benr           379K   199M   355K  /backups/home/benr
backups/home/benr@001        24K      -    88K  -
backups/home/benr@002          0      -   355K  -
backups/home/benr@nov6         0      -   355K  -
backups/home/conradr         88K   199M    88K  /backups/home/conradr
backups/home/conradr@nov6      0      -    88K  -
backups/home/glennr          88K   199M    88K  /backups/home/glennr
backups/home/glennr@nov6       0      -    88K  -
backups/home/novar           88K   199M    88K  /backups/home/novar
backups/home/novar@nov6        0      -    88K  -
backups/home/tamr            88K   199M    88K  /backups/home/tamr
backups/home/tamr@nov6         0      -    88K  -

Simple, just snapshot the parent dataset with the -r flag, then send the parent dataset snapshot with the -R flag. Otherwise, its all the same! And, of course, you can combine this with all our other pipe tricks just the same!

And so we see that using a single set of commands, we have simplistic and powerful replication, backup, and archive capabilities. A lot of power unleashed with just a little imagination; thats the ZFS way.


- - C O M M E N T S - -

I’m not sure, but I think I want ZFS to have my baby

Matt Simmons (Email) (URL) - 07 November '08 - 00:36

These posts are just fantastic, definitely great for Linux people like me who are OpenSolaris-curious and have 2008.5 installed on a spare partition… keep them coming!

Pedro (Email) - 07 November '08 - 06:19

Some kind of synchronous data replication based on zfs would be nice. This is often a requirement at places where fault tolerance is more important than high availability (or performance).

Robert - 07 November '08 - 13:20

Robert: Look at AVS: [[http://www.opensolaris.org/os/project/..]]

benr - 07 November '08 - 17:21

ZFS may be the coolest thing
SINCE sliced bread

JV - 09 November '08 - 05:55

The method you describe would be good for replacing licensed backup software. However I was wondering if it was possible to do this from within the Fishworks GUI ? From what I can gather, Fishworks doesn’t allow for access to a Solaris shell that can operate on the underlying filesystems.

Jeroen (Email) (URL) - 17 November '08 - 00:05

Yeah that’s cool part of ZFS – but what’s about scalability? For ex. I have a thousand of children filesystems and want to sync them all to backup server with -R. Have you tried something like that? I’m still trying but… it works terrible for me. Any suggestions? Thanks.

Mike (Email) - 20 November '08 - 22:26

Question regarding ZFS backups. I have ZFS pool (2.3x compression), when i backup a 100MB file becomes 300MB file on tape. Is there a better way to backup this data. The whole ZFS filesystem is quite large about 30TB used, and i afraid if the backup goes this way it will end up being 80TB on tape.

Any ideas/suggestions are really appreciated.

Thanks

Nav (Email) - 18 April '09 - 00:44

These posts are just fantastic, definitely great for Linux people like me who are OpenSolaris-curious and have 2008.5 installed on a spare partition.

Chris (Email) (URL) - 10 May '09 - 07:41

@Mike

At risk of stating the obviously which you likely know, but for the benefit of the uninitiated, given zfs conducts it’s own compression, when you transport/pipe it outside of the zfs filesystem to a tape device, for example, you have two immediate options: 1) pipe the raw data to the tape’s device with compression togged on or 2) pipe it through a compression utility beforehand like gzip and then to the tape device (with compression off or course).

Great post Ben.

Daniel Gomez (Email) (URL) - 05 June '09 - 15:29

It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader. Thanks again.

Christian louboutin shoes (Email) (URL) - 27 October '09 - 03:14

Thank you very much!

cheap links of london (Email) (URL) - 07 November '09 - 00:57

Great post! Hope to be better. Better means more features.
good post,I think so!
Thanks for your information, i have read it, very good!
Bing is a really overlord!! support Bing~~
This is great news. Best of luck for the future and keep up the good work.

links of london (Email) (URL) - 17 November '09 - 03:25

Links means care, like and love which emotion is owned by people.

liubaiying (Email) (URL) - 31 December '09 - 07:35

Just one question: how to add your blog into my rrs reader, thanks so much.

christian louboutin (Email) (URL) - 03 January '10 - 07:02

This may be a perfect example of information asymmetry and adverse selection in insurance. I hope all textbook authors and legislators notice.

christian louboutin shoes (Email) (URL) - 06 January '10 - 08:50

good idea!

christian louboutin (Email) (URL) - 08 January '10 - 09:02

thank you very much

links of london (Email) (URL) - 12 January '10 - 04:02

Good post! Thanks you for your information! China Wholesale Wholesale China Wholesalers Wholesale Game Accessories Wholesale Iphone Accessories Video Game Accessories Wholesale Wholesale Wii Accessories Wholesale Xbox 360 Accessories Wholesale Xbox 360 Games Wholesale Video Games Cheap Video Games Cheap Ps3 Games Cheap Xbox 360 Games Wholesale Computers Wholesale Laptop Computers Wholesale Laptops Discount Computers Cheap Computers Wholesale Iphones Wholesale Iphone Wholesale Iphones 3g Hiphones Wholesale Hiphone Wholesale Hiphones Wholesale Nokia Wholesale Nokia 8800 Wholesale Nokia n97 wholesale blackberry wholesale blackberry phones wholesale blackberry 9700 wholesale blackberry 9600 wholesale blackberry 9500

raging bull (Email) (URL) - 15 January '10 - 06:42

thanks

kamagra (Email) (URL) - 21 January '10 - 01:21

good read thanks

M65 Jacket (Email) (URL) - 21 January '10 - 01:21

many thanks

viagra cialis (Email) (URL) - 21 January '10 - 01:55

It is very great thing for the posts, thank you for sharing.

charms (Email) (URL) - 22 January '10 - 07:21

discount links of london [[http://www.linksoflondonhut.co.uk]]
links of london bracelets [[http://www.linksoflondonhut.co.uk/Link..]]

links of London (Email) (URL) - 22 January '10 - 08:22

Ben,

Great post on using ZFS send/recv for replication of ZFS datasets. When I first started my quest for replication of ZFS, I ran into AVS, which is quite impressive, however, definitely not easy to configure. In addition, I think hardware similarity is required between the two units. I wanted to replicate ZFS datasets to another storage device that had differing hard drive sizes in the ZFS pool.

In your article you mentioned being creative and creating a script to enable frequent replications between remote systems. This was the idea I was thinking of when I found your blog post. The difficulty I am having is process/logic to use for replicating that often, say every 30 seconds. To send/recv incrementally, you have to specify two snapshots. From what I can tell, those two snapshots must be the currently replicated snapshot level on the target system and the most recent (or more recent) snapshot on the source system. Unless I am misunderstanding, the kicker is that the “currently replicated” snapshot must reside on both systems.

How do you manage data requirements for that many snapshots (2880/day at 30 seconds) on a device that has 2TB of always changing data, such as in my situation which is an iSCSI target for VMware hosts. And if you do decide to manage the snapshots, by removing them I guess, how would the script know which snapshot to use? If the dataset on the target contains different snapshots than the one of the source, you may have to delete the ZFS dataset and start over.

Just some thoughts—- I will probably have a ZFS replication script on my blog at some point in the near future, with, hopefully, these issues resolved.

-Aaron

Aaron Gilbert (URL) - 22 January '10 - 23:03

I liked your post, I will bookmark your site and follow you!

lazy boy kids furniture (Email) (URL) - 23 January '10 - 22:58

New battery review module surface is black, not because of the positive electrode exists, the module is likely to be integrated back contact silicon solar cells, silicon, where the role is often to extend the service life of equipment.Mitsubishi motors in October 2009 28-30 at the yokohama exhibition center held the Pacific “GreenDevice2009″ displayed on the high efficiency solar camera battery 19.1% respectively. In the square 15cm in polysilicon solar cells, realized the highest efficiency. The research achievements in 2009 September 2009 and the application of physics to the international society in October EUPVSEC “, “said in a statement.

battery review (Email) (URL) - 30 January '10 - 03:39

Ed Hardy Hats were later launched with much fanfare.

ed hardy (Email) (URL) - 25 February '10 - 03:45

These beautiful jewelries can catch the eyes of women all the time since they always believe that the jewelries in the shop window are much better than those in their jewelry box.

links of london (Email) (URL) - 04 March '10 - 06:27

These beautiful links of londonjewelries can catch the eyes of women all the time since they always believe that the jewelries in the shop window are much better than links of londonthose in their jewelry box.

links (Email) (URL) - 04 March '10 - 06:28

Your site is a refreshing change from the majority of sites I have visited. When I first started visiting web sites I was excited by the potential of the internet as a resource and was very disappointed initially. You have restored my enthusiasm and I thank you for your efforts to share your insights and help the world become a better place.

wholesale lingerie (Email) (URL) - 10 March '10 - 10:40

海湾每日相伴的女孩。看着女孩的苍白的脸,男孩哭了。他哭了很伤心。他坐在除了每晚的女孩,并开始折叠的纸起重机。因此,几个月通过。那女孩是她最后奄奄一息,毫无知觉的时候。这名男孩还是折叠的一天,她一天纸鹤。当他的第1000折,房间里灯火通明,一个天使突然在前面的男孩出现。该名男童僵化,摸摸自己的眼睛是红色的,持有女孩的手,哭的天使“,请帮助这个善良的姑娘”天使的孩子说,“如果你喜欢,你是3年蜻蜓,女童会一天天好慢慢来“,没有犹豫承诺的男孩。天使消失了,男孩已经变成了在同一时间蜻蜓。

joejonas (Email) (URL) - 26 March '10 - 03:14

Christian louboutin sale ladies are always looking for christian louboutin collections among UK, and we can get access to the christian louboutin discount like christian louboutin shoes and cheap christian shoes on the int

christian louboutin (Email) (URL) - 01 April '10 - 09:07

That’s interesting. polo T-shirts

Danny (Email) (URL) - 12 April '10 - 03:26

Cool. [[http://www.fashion-sale1.com]]

saylor (Email) (URL) - 12 April '10 - 03:27

That’s cool. [[http://www.christianlouboutinbigmall.c..]]

Danny (Email) (URL) - 13 April '10 - 04:09

abercrombie
abercrombie hoodie

abercrombie (Email) (URL) - 19 April '10 - 19:30

I very intersted in the article

cheap mbt shoes (Email) (URL) - 23 April '10 - 03:27

Tags: links of London jewelry, links of London Bangles, links of London Necklace, links of London Pendant, links of London Bracelets, links of london links of London sweetie bracelet, links of London Earrings, links of London Rings, links of London Charms, links of London sale

wangrongjiao (Email) (URL) - 28 April '10 - 08:15

sale supra cuban

supra shoes (Email) (URL) - 29 April '10 - 09:03

I think he’s probably trying to get himself right. I’m all supportive of that.” “I listened to coaches, fans, people on campus, the board, the president and I looked at our history with the goal of bringing Michigan State together to project a unified Nike sb dunk.”“There’s no Nike-forced conspiracy on these changes,” Hollis said. Welcome to shop [[http://www.nikedirect.net/.]].

Nike shock shoes (Email) (URL) - 03 May '10 - 09:16

P90x .It really is not expensive if you factor in the cost

of a gym membership,P90x workout . The cost for P90X is

about three months of a paid gym membership but you get to

keep the program foreverP90x . You can try many of the

online sites, but it will be the same as buying from the

company or a Beachbody Coach. Make sure you are getting

original DVD’s. People are selling copies all over. The

problem is how long will they last, P90x workout ,and you

truly need the exercise and nutrition guide to even follow

the program. You can go to any site or you can go to

[[http://www.p90xmall.com]] and click on products. P90x dvd You can

order directly from the site,P90x dvd.

p90x (Email) (URL) - 05 May '10 - 05:47

Generally speaking, if the clothing is very simple, there is no decoration, you can wear a brooch or pin. links of london Brooch can be pinned to the collar on a jacket or suit, and the shoulders. However, if wearing a, it can only be worn on the inside of jacket, visible only from the neck, but not worn on the outside.

wangrongjiao (Email) (URL) - 06 May '10 - 07:13

Nike air max have utilized the technology of air in it’s sole to give us a more comfortable and supportive cushion to walk on.You may also love nike shox and it is natural to attract most consumers.Waiting for the shoe store to open up so you can get your hands on the first pair of nike shoes what you like. [[http://www.airmax-online.com/]]

Online products (Email) (URL) - 06 May '10 - 09:38

Generally speaking, if the clothing is very simple, there is no decoration, you can wear a brooch or pin. links of london Brooch can be pinned to the collar on a jacket or suit, and the shoulders. However, if wearing a, it can only be worn on the inside of jacket, visible only from the neck, but not worn on the outside. [[http://www.towatches.com/Discount-Watc..]]

wholesale watch faces (Email) (URL) - 07 May '10 - 06:37

I first knew christian louboutin in a film, but I don’t remember the name already.And I wish to have the christian louboutin shoes as well.However where christianlouboutin.

Avins (Email) - 10 May '10 - 06:47

Generally speaking, if the clothing is very simple, there is no decoration, you can wear a brooch or pin. links of london Brooch can be pinned to the collar on a jacket or suit, and the shoulders. However, if wearing a, it can only be worn on the inside of jacket, visible only from the neck, but not worn on the outside.
www.christianlouboutincollection.com

wert (Email) - 10 May '10 - 06:49

My working partner Dandan, who is the same age as me, got married after I married the same year. links of london jewellery And two months later when my baby was born, she also gave birth to a daughter called xinxin.

wangrongjiao (Email) (URL) - 10 May '10 - 09:29

hi there, thanks for your great post, it help me so much! now i wonder if you linke the mbt shoes from my site: the url is [[http://www.thesuitshoes.com]]

mbt shoes (Email) (URL) - 12 May '10 - 07:04

Legend of Pandora Jewelry can have a magic spell, Because she was a symbol of good, so if you bring any of a pandora ring charm you pandora will not get back once lost himself, get back the lost happiness from the new, recovered all had good, If we had love go, be like, even if the Pandora rings, still wearing my hand, I lost you, of course, Pandora is destined to be a symbol of greedy, so you and pandora charms store locator only one destined to accompany beside me, so I lose you after all, to lose that part of Sentimental love, lose it all, is to the happiness I want, Pandora’s blessing or not I stay here

wangrongjiao (Email) (URL) - 14 May '10 - 08:20

Dell d830 Battery [[http://www.adapterlist.com/dell/d830.h..]]

laptop battery (Email) (URL) - 15 May '10 - 05:59

Dell d620 Battery [[http://www.globallaptopbattery.co.uk/d..]]

laptop batteries (Email) (URL) - 15 May '10 - 06:02

MBT Masai Barefoot Technology.The on behalf of the company, making shoes, main battle tanks ‘also known as anti-shoe’ is a Swiss Masai.MBT Shoes unique design, multi-level, the only cure is designed to simulate walking in sand ‘similar to walking in the natural environment in Kenya Masai’. This creates a natural, uneven walking surfaces, and use of physical force and stability of the major muscle groups, training the body to move correctly.It has a positive effect on the whole body,help improve posture, reuce back pain, and alleviate joint pressure. MBTs will be your best healthy life partner.

mbt shoes (Email) (URL) - 22 May '10 - 13:43

Buy Nike Air Max 90 Shoes just $45-55 USD in[[http://www.iofferitems.com,]], 40-70% Off. Cheap Air Max 90 Shoes, Free Shipping! Buy Air Max 90 Now!

nike air max 90 shoes (Email) (URL) - 28 May '10 - 02:22

Buy Nike Air Max 90 Shoes just $45-55 USD in [[http://www.iofferitems.com,]], 40-70% Off. Cheap Air Max 90 Shoes, Free Shipping! Buy Air Max 90 Now!

nike air max 90 shoes (Email) (URL) - 28 May '10 - 05:37

thank you for sharing the article.
The easy-to-use Video Converter for Mac lets you to enjoy your videos on all sorts of palyback including PSP, iPod, Mobile Phone, Zune, iPhone, Apple TV and MP4/MP3 player.
Free download supported. [[http://www.videoconverterformac.com]]

video converter for mac (Email) (URL) - 28 May '10 - 05:53

7 days herbal slim7 days herbal slim

Super slimSuper slim

lose weight (Email) (URL) - 29 May '10 - 07:21

[[http://www.sxhgz.com]] : nike shoes

puma shoes (Email) (URL) - 21 June '10 - 01:57

[[http://www.mbtshoeslatest.com]]
[[http://www.nikeairmaxshoe.com]]

Nike air max shoe (Email) (URL) - 24 June '10 - 07:59

Do you like the ugg ? Ugg Classic Short Boots are UGG Australia’s original heritage styles. Ugg Classic Tall will keep your feet dry and the ultimate comfortable.
[[http://www.ugg-boots-london.com/ugg-cl..]]

Ugg Classic Short (Email) (URL) - 07 July '10 - 01:24

Good post, thanks for sharing, I enjoy it very much! There is a site: [[http://www.cheapinsanitydvds.com/]]

Insanity program (Email) (URL) - 07 August '10 - 13:39

The first layer of MBT Shoes means a high quality MBT Footwear cowhide after processing of machine. The main feature of MBT first layer cowhide: Masai Shoes surface layer are made of grain materials with close woven fibers; MBT Sandals feature smooth feeling, good strength and great abrasive resistance. MBT Medical Shoes lining are composed of suede with thick fiber, big diameter and flocking wool on the surface. Comparing with second layer of Discount MBT Sheos, first layer of MBT Shoes Cheap with more smooth surface and better quality, can be worn longer time. Our website launches MBT Outlet with first layer cowhide, so you can feel free to shop here and get satisfied MBT Shoes Clearance, including MBT Sawa, MBT Sirima, MBT Kaya, MBT Changa and so on .
Welcome to our website: [[http://www.mbtshoesmasai.com]]

mbtshoesmasai (Email) (URL) - 09 August '10 - 02:52

Buy a piece of ghd for yourself. Come and join us [[http://www.ghdiron-outlet.com/]] to win the cheap ghd.

ghd outlet (Email) (URL) - 13 August '10 - 03:19

As we know, now GHD are loved by more and more people, which will save up to 45%.welcome to [[http://www.ghdoutlet-au.com/.]].

ghd straighteners (Email) (URL) - 13 August '10 - 03:21

Come and join us [[http://www.ghdoutlet-uk.com/index.php]] to win the ghd iv styler.

ghd hair straighteners (Email) (URL) - 13 August '10 - 03:21

Your post is awesome, but why not take a look at our site: [[http://www.lovemypursemall.com]]

designer bags (Email) (URL) - 14 August '10 - 14:34

very cool article ,thanks for sharing the article!like my cool stuff .very useful.
uCoolStuff is the leading China wholesaler for [[http://www.ucoolstuff.com]] cool stuff , [[http://www.ucoolstuff.com]] cool gifts, unusual gadgets and other unique gift ideas. We provide the very latest cool stuff and cool gifts for you

cool stuff (Email) (URL) - 25 August '10 - 08:36

excellent article , I added you to my [[http://www.china-wholesale-directory.c..]] Top China Wholesalers category.. thanks for sharing the article!

China Wholesale Directory (Email) (URL) - 25 August '10 - 08:41

I’ve read that using “zfs send” to tape is quite dangerous: small corruption renders the file unreadable by zfs receive.( [[http://www.solarisinternals.com/wiki/i..]] )

To circumvent this, I’m creating a replication pool in a file vdev:
mkfile -n 500g /backups/filepool.vdev
zpool create filepool /backups/filepool.vdev
zfs set compression=gzip-9 filepool
zfs send data/myfs@today|zfs receive filepool/myfs
zpool export filepool

now the file /backups/replpool.vdev can be stored to tape as a monthly backup file.
This way, I hope that in case of (small) corruption ZFS will be able to keep parts with good checksums.
An added benefit is that the tape can be used at its full speed.
restore is very fast too.
ZFS gzip compression is a lot faster than the gzip command itself on T5220.
To spare tape space, I’m doing incremental zfs send for the dailies.

My only question is whether in case of emergency, I can start a production quality service from a file vdev and replicate/mirror it to the real storage afterwards.
I’ve read somewhere that you should’nt use file vdev for other purpose than testing ( although iscsi volumes are generaly also implemented as big files…).

Michel

Michel Jansens - 31 August '10 - 07:00

Top Online Stores is a SEO Friendly [[http://www.toponlinestores.org]] free directory where you can find the best online shopping stores selected by hand and sorted by category [[http://www.china-wholesale-directory.c..]] china wholesale .

link directory (Email) (URL) - 14 September '10 - 07:31

[[http://pageantvilla.com/q4c/index.php?..]]

sw (Email) - 14 September '10 - 09:42

There are various kinds of ghd hair straighteners on hot sell. GHD styler specialized in ghd australia market and ghd outlet. It provides many cheap ghd, ghd iv styler, pink ghd, purple ghd, which save up to 50%, with free shipping!

ghd outlet (Email) (URL) - 25 October '10 - 05:19

00zoe

MBT shoes[[http://www.mbtshoes4sale.co.uk]]
Timberland boots[[http://www.timberlandbootsale.co.uk]]

MBT shoes clearance (Email) (URL) - 01 November '10 - 03:08

One day they will understand you.

Anything in the Amazing Electronic Gadgets World (Email) (URL) - 16 December '10 - 08:47

Personal information





Remember your information?
Comment

Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.


^M