Understanding ZFS: Disk Space Discrepancies
21 Jan '09 - 09:53 by benrHere's a good ZFS Trivia question to bewilder your friends and common question: Why do the two following outputs for the same ZFS pool disagree?
# zfs list zones NAME USED AVAIL REFER MOUNTPOINT zones 634G 185G 69K /zones # zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT zones 832G 634G 198G 76% ONLINE -
If you add up USED and AVAIL from the zfs list output you get 819GB. So why does zfs list say we have 819GB but zpool list says we have a 832GB pool?
This is a question I have tried to answer in the past using zdb, quite unsuccesfully. I found that answer while digging through the ZFS code (dsl_pool.c):
408 uint64_t
409 dsl_pool_adjustedsize(dsl_pool_t *dp, boolean_t netfree)
410 {
411 uint64_t space, resv;
412
413 /*
414 * Reserve about 1.6% (1/64), or at least 32MB, for allocation
415 * efficiency.
416 * XXX The intent log is not accounted for, so it must fit
417 * within this slop.
418 *
419 * If we're trying to assess whether it's OK to do a free,
420 * cut the reservation in half to allow forward progress
421 * (e.g. make it possible to rm(1) files from a full pool).
422 */
423 space = spa_get_dspace(dp->dp_spa);
424 resv = MAX(space >> 6, SPA_MINDEVSIZE >> 1);
425 if (netfree)
426 resv >>= 1;
427
428 return (space - resv);
429 }
So 1/64th of the pool is reserved? It makes sense, a copy-on-write filesystem is in trouble if it truly hits 100% Used. But do those numbers fit?
1/64th of ZPool: 832G * 0.016 = 13.3GB Output Discrepency: 832G - 819GB = 13GB
Right on the money.
Therefore, let it be known! Above the capacity lost to whatever RAID scheme you choose, you will loose 1/64th of the pool to reserve. Based on prior experiments, I can verify definitively, when you hit 100% according to zfs list you can not write any more data. Therefore, never report or monitor ZFS capacity based on zpool list! As a point of interest, df also invokes this adjustedsize function and therefore outputs correct numbers. If your monitoring system tracks disk capacity based on df no change is needed.
Very nice find. I’ve seen similar effects on different filesystems, and it seems that each filesystem has its own reason for not always accurately reporting disk space. Good to know the underlying reason, and another benefit of open source software.
Matt Simmons (Email) - 21 January '09 - 12:57
I used to monitor space with zpool until 2w ago… then it said 97% full and still a few GBs available but zfs list reported less than half of that was free. WTF?! Now it makes sense: 3% free – 1.6% reserved. Thanks ben.marcelo - 21 January '09 - 14:24
Zpool free is also misleading reporting free space with raidz, so it’s looking like zfs list (and calculating the percentage used/free manually) is the way to go in all cases. Interestingly, the zpool man page mentions reserving space ‘for internal accounting’, but doesn’t go into any more detail, not mention how much is reserved.Mark Harrison - 21 January '09 - 16:21
Thta was an inspiring post,its a very informative article, keep posting, and dont drop the good work…
Thanks
software development london (Email) (URL) - 14 October '09 - 15:58
Great post! Hope to be better. Better means more features.good post,I think so!
Thanks for your information, i have read it, very good！
Bing is a really overlord!! support Bing~~
This is great news. Best of luck for the future and keep up the good work.
tiffany and co (Email) (URL) - 19 November '09 - 01:38
[url=[[http://www.inuggshopping.com]UGG]] BOOTS[/url] Discount40%-50% offfree shipping
Discount40%-50% off
UGGBOOTS-free shipping! Discount40%-50% off (Email) (URL) - 30 December '09 - 07:45
Yes,I very like your article,and they’re gorgeous!Thank you to share this!At the same time I also like christian shoes,because they are design for femails and they are sobeautiful.In my opinion,they are what you can not miss.They are your best choice .
christian louboutin boots (Email) (URL) - 11 January '10 - 08:41
many thanksM65 Jacket (Email) (URL) - 21 January '10 - 02:01
New battery review module surface is black, not because of the positive electrode exists, the module is likely to be integrated back contact silicon solar cells, silicon, where the role is often to extend the service life of equipment.Mitsubishi motors in October 2009 28-30 at the yokohama exhibition center held the Pacific “GreenDevice2009″ displayed on the high efficiency solar camera battery 19.1% respectively. In the square 15cm in polysilicon solar cells, realized the highest efficiency. The research achievements in 2009 September 2009 and the application of physics to the international society in October EUPVSEC “, “said in a statement.battery review (Email) (URL) - 30 January '10 - 03:38
nice post here, if you want to know more about best mbts and ugg boots, just click here[[http://www.bestmbtshoes.com]]
[[http://www.uggbootsroom.com]]
best mbt (Email) (URL) - 06 February '10 - 03:28
nowGoogle.com adalah Multiple Search Engine Popular memiliki Fungsi Multiple Search Engine hadir di Catatan Si Bongo dengan nama nowGoogle.comnowGoogle.com adalah Multiple Search Engine Popular (Email) (URL) - 09 March '10 - 16:58
It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader.manolo shoes (Email) (URL) - 31 March '10 - 11:07
The aim of Adidas shoes is to make the very best adidas superstar shoes at an affordable price.cheap Adidas shoes of running sneakers is about a lifestyle. Always making style and innovative advances, cheap Adidas shoes have the fashionable running shoes you will love.From inexpensive adidas running shoes to competitive track shoes, the Adidas superstar sneakers offer great value.[[http://www.adidas-cheap.com/]]
adidas shoes (Email) (URL) - 08 April '10 - 10:07
To recommend a wonderful puma store [[http://www.pumashoesale.com]] , where you can find the perfect puma ferrari shoes. Nothing is impossible, come on, to get the best puma sneakers, enjoy shopping.puma shoes (Email) (URL) - 08 April '10 - 12:28
i agree with youghd hair straighteners (Email) (URL) - 19 April '10 - 02:59
I very intersted in the articlecheap mbt shoes (Email) (URL) - 23 April '10 - 03:40
fegthyhkuksupra shoes (Email) (URL) - 28 April '10 - 09:06
great post[[http://www.sellnikesbs.com]]
mbt sheos (Email) (URL) - 01 May '10 - 01:54
It was a very nice idea! Just wanna say thank you for the information you have shared.[[http://www.trademic.com]]
wholesale (Email) (URL) - 04 May '10 - 03:22
discount mbt shoes [[http://www.thesuitshoes.com]] mbt shoes on sale.www.thesuitshoes.com (Email) (URL) - 05 May '10 - 03:26
P90x .It really is not expensive if you factor in the costof a gym membership,P90x workout . The cost for P90X is
about three months of a paid gym membership but you get to
keep the program foreverP90x . You can try many of the
online sites, but it will be the same as buying from the
company or a Beachbody Coach. Make sure you are getting
original DVD’s. People are selling copies all over. The
problem is how long will they last, P90x workout ,and you
truly need the exercise and nutrition guide to even follow
the program. You can go to any site or you can go to
[[http://www.p90xmall.com]] and click on products. P90x dvd You can
order directly from the site,P90x dvd.
p90x (Email) (URL) - 05 May '10 - 06:00
for best web online store [[http://www.salepumashoes.com]]puma (Email) (URL) - 08 May '10 - 12:46
china wholesale Computer Peripheral[[http://www.iaamart.com/computer-periph..]]
china wholesale Computer Peripheral (Email) (URL) - 11 May '10 - 09:49
A good idea! [[http://www.offerreplicawatches.com.The..]] sell Chanel Watches, Replica BRM Watches,offer Cartier Watches, Copy Ebel Watches, Replica Hublot Watches, Fake Tag Heuer Watches, wholesale Richard Mille Watches.They also sell some very fashion watches, such as Montblanc Watches for sale,Chopard Watches, Dior Watches and Gucci Watches. Rolex watches are hot sell. If u are Armani Watches and Breitling Watches fan. You can still look at Omega Watches and Longines Watches, many new arrivals watches just come here.You can have a try on 58338312391273913. :[url=[[http://www.offerreplicawatches.com]www..]]].
Replica Chanel Watches,Dvd Boxes (Email) (URL) - 12 May '10 - 07:39
thank you for this outstanding article.I thought certino was the best technologh for laptop battery performance.We specialize in [[http://www.adapterlist.com/dell/gd761...]] ,laptop AC adapters. All our products are brand new, with the excellent service from our laptop battery of customer service team.laptop battery (Email) (URL) - 14 May '10 - 08:59
If you are looking for the best brand new laptop batteries at the most preferred price, [[http://www.globallaptopbattery.co.uk/s..]] you have come to the right place. We provide the highest quality Battery at the lowest price with the highest level of service, all in a secure and convenient platform.laptop batteries (Email) (URL) - 14 May '10 - 09:02
It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader. [url=[[http://www.mydiscountjordanshoes.com/a..]]]] air-jordan-xv[/url]
air jordan xv (Email) (URL) - 15 May '10 - 03:49
Buy Nike Air Max 90 Shoes just $45-55 USD in[[http://www.iofferitems.com,]], 40-70% Off. Cheap Air Max 90 Shoes, Free Shipping! Buy Air Max 90 Now!nike air max 90 shoes (Email) (URL) - 28 May '10 - 02:26
[[http://www.vibramfivefinger.us/]] vibram five fingersvibram five fingers (Email) (URL) - 10 June '10 - 09:07
Great read. I enjoyed reading your post and I like your take on the issue. Thanks.dentist (Email) (URL) - 15 June '10 - 06:44
Great read. I enjoyed reading your post and I like your take on the issue. Thanks.[[http://www.cosmeticdentistryguide.co.u..]]
dentist (Email) - 15 June '10 - 06:46
[[http://www.fivefingersvibram-shoes.com..]]adidas-cheap.com (Email) (URL) - 20 June '10 - 11:30
[[http://www.mbtshoeslatest.com]][[http://www.nikeairmaxshoe.com]]
Nike air max shoe (Email) (URL) - 24 June '10 - 06:49
American classic leather goods brand Coach, the style of which is simple and durable and are popular with most people. Coach Bags has good reputation, which are embraced by the high society and the common people. The original inspiration of designing Coach Handbags is from one softball glove. The first Coach Brand founder, Miles caha watch the softball game and found the special features of its gloves Coach Handbags 2010, which is convenient and durable and then he design Coach bags at once. As a result, it is popular by most consumers. Furthermore, these Cheap Coach Bags are at a competitive price which can easily accept by large people and now there are many Coach Outlet set up among many countries so you can buy it very easily.[[http://www.bag-salon.com]]
coach bags (Email) (URL) - 04 July '10 - 08:07
Do you like the ugg ? Ugg Classic Short Boots are UGG Australia’s original heritage styles. Ugg Classic Tall will keep your feet dry and the ultimate comfortable.[[http://www.ugg-boots-london.com/ugg-cl..]]
Ugg Classic Short (Email) (URL) - 07 July '10 - 01:14
Forever 21, the US clothes retailer popular for selling fashion-forward clothing at low prices, has courted controversy by expanding into maternity wear. The brand, loved byyoungsters and teenagers, has launched Love 21, a range of cheap and stylish maternity clothing including jeans for as little as US $8.
[[http://www.ed-hardy.cc/ed-hardy-clothi..]] ed hardy clothes
[[http://www.ed-hardy.cc/ed-hardy-caps.h..]] ed hardy hat