Thump Thump

[thumper3:/splash] root# time dd if=/dev/zero of=testfile bs=1024k &
[thumper3:/splash] root# zpool iostat 1
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
splash      11.2G  20.0T     96    289  11.3M  34.6M
splash      11.2G  20.0T      0  4.15K      0   531M
splash      11.2G  20.0T      0  3.98K      0   509M
splash      11.2G  20.0T      0  3.99K      0   511M
splash      18.2G  20.0T      0  1.56K      0   101M
splash      18.2G  20.0T      0      0      0      0
splash      18.2G  20.0T      0      0      0      0
splash      18.2G  20.0T      0      0      0      0
splash      18.2G  20.0T      0    727      0  90.9M
splash      18.2G  20.0T      0  3.42K      0   438M
splash      18.2G  20.0T      0  4.21K      0   539M
splash      18.2G  20.0T      0  4.19K      0   536M
splash      18.2G  20.0T      0  4.00K      0   511M
splash      18.2G  20.0T      0  3.71K      0   475M
splash      18.2G  20.0T      0  3.83K      0   490M
splash      18.2G  20.0T      0  3.86K      0   494M
splash      18.2G  20.0T      0  3.78K      0   484M
splash      18.2G  20.0T      0  3.74K      0   479M
splash      18.2G  20.0T      0  3.72K      0   477M
splash      18.2G  20.0T      0  4.20K      0   537M
splash      25.1G  20.0T      0  2.05K      0   164M
^C

Half a gig a second? Not bad. Too bad I still can’t get the iSCSI Target to push a remotely decent amount of throughput, not to mention the memory suckage:

[thumper3:/] root# prstat
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
   458 root      100G 7112M cpu2     0    0   0:01:01  22% iscsitgtd/18
...

…back to work.

11 Responses to “Thump Thump”

  1. Of course you realize dd as even a cheezy benchmark tool is only interesting for testing bandwidth if you have multiple threads (processes in this case) going at once. It’d be more interesting to see 8 or 10 of the suckers going at once.

    Dunno about the iscsi target thing. What’s the pmap look like for the daemon? There might be a rational explanation somewhere.

  2. Lasse says:

    iscsitgtd mmap’s the entire size of it’s target(s), at least that’s the case when using filesystem based backingstores.

    What sort of throughput are you seeing and in which scenario? I’ve been playing around with it as well and haven’t been able to push much through either.

  3. 卡诫魔 says:

    Come to your blog just by chance! I am a CHINESE boy. best wish you come back to my space , thanks! 你是我进入的第一个国外blog,欢迎你来我的空间,谢谢!

  4. Hambali says:

    Assuming you’re using a set of channeled Gig-E interfaces, you’ll be limited to only single port speed as both Etherchannel and LACP don’t really trunk the interfaces together for a single IP.

    Both Etherchannel and LACP (to a lesser degree) use an algorithm to determine which interface of a channel will be used for a particular IP/TCP port based upon the last digit in the MAC address.

    If you’re looking for high bandwidth, you should try a test with multiple iSCSI targets (i.e. different IP’s). Not sure if this is guaranteed to work as the channeling algorithm may only look at the server’s MAC which is mounting the LUNs.

    To take advantage of port channelling you really need to have a large mass of server going to the channelled interface. Then it will look like you’re getting 4x bandwith (or whatever the channel size is).

    BTW, sequential I/O is a horrible test (as pointed out by Matt). What writes files in 1024 block sizes? Try something like 8k randoms (Database), or 32k sequentials for real world (copies).

  5. zithromax says:

    shop zithromax welcome

  6. Trudy says:

    Very interesting article: “Thump Thump”