# [OmniOS-discuss] zfs pool 100% busy, disks less than 10%

Rune Tipsmark rt at steait.net
Mon Nov 3 02:24:29 UTC 2014

- Previous message: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
- Next message: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Looking a bit more at these numbers, am I seeing twice the actual rate due to mirroring?
    How does compression affect the numbers?

    Say I have 1 vdev mirrored and I see the pool writing 100 MB/sec, its what? 50 MB each disk? But from the client side only 50 MB/sec total? What if I compress it at the same time at say 1.50 ratio, will the pool show 100 MB/sec and the client write 75 MB/sec actual?

    Br,
    Rune

    -----Original Message-----
    From: Richard Elling [mailto:richard.elling at richardelling.com]
    Sent: Sunday, November 02, 2014 6:07 PM
    To: Rune Tipsmark
    Cc: Eric Sproul; omnios-discuss at lists.omniti.com
    Subject: Re: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%

    On Oct 31, 2014, at 6:07 PM, Rune Tipsmark <rt at steait.net> wrote:

    > So actually started storage vmotions on 3 host, 6 concurrent and am
    > getting about 1GB/sec Guess I need more hosts to really push this, the disk are not more than 20-25% busy, so in theory I could push a bit more.
    >
    > I think this is resolved for now.... cpu sitting at 30-40% usage while
    > moving 1GB/sec

    Yes, that seems about right.
     -- richard

    >
    > Iostat -xn 1
    > pool04       396G  39.5T      9  15.9K   325K  1.01G
    > pool04       396G  39.5T      7  17.0K   270K  1.03G
    > pool04       396G  39.5T     12  17.4K   558K  1.10G
    > pool04       396G  39.5T     10  16.9K   442K  1.03G
    > pool04       397G  39.5T      6  16.9K   332K  1021M
    > pool04       397G  39.5T      1  16.3K  74.9K  1.01G
    > pool04       397G  39.5T      8  17.0K   433K  1.05G
    > pool04       397G  39.5T     20  17.1K   716K  1023M
    > pool04       397G  39.5T     11  18.3K   425K  1.14G
    > pool04       398G  39.5T      0  18.3K  65.9K  1.11G
    > pool04       398G  39.5T     16  17.9K   551K  1.06G
    > pool04       398G  39.5T      0  16.8K   105K  1.03G
    > pool04       398G  39.5T      1  18.2K   124K  1.11G
    > pool04       398G  39.5T      0  17.1K  45.9K  1.05G
    > pool04       399G  39.5T      6  17.3K   454K  1.08G
    > pool04       399G  39.5T      0  17.9K      0  1.06G
    > pool04       399G  39.5T      2  16.9K   116K  1.04G
    > pool04       399G  39.5T      2  18.8K   130K  1.09G
    > pool04       399G  39.5T      0  17.6K      0  1.03G
    > pool04       400G  39.5T      3  17.5K   155K  1.04G
    > pool04       400G  39.5T      0  17.6K  31.5K  1.03G
    >
    > -----Original Message-----
    > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com]
    > On Behalf Of Rune Tipsmark
    > Sent: Friday, October 31, 2014 12:38 PM
    > To: Richard Elling; Eric Sproul
    > Cc: omnios-discuss at lists.omniti.com
    > Subject: Re: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
    >
    > Ok, makes sense.
    > What other kind of  indicators can I look at?
    >
    > I get decent results from DD but still feels a bit slow...
    >
    > Compression lz4 should not slow it down right? Cpu is not doing much when copying data over, maybe 15% busy or so...
    >
    > Sync=always, block size 1M
    > 204800000000 bytes (205 GB) copied, 296.379 s, 691 MB/s
    > real    4m56.382s
    > user    0m0.461s
    > sys     3m12.662s
    >
    > Sync=disabled, block size 1M
    > 204800000000 bytes (205 GB) copied, 117.774 s, 1.7 GB/s
    > real    1m57.777s
    > user    0m0.237s
    > sys     1m57.466s
    >
    > ... while doing this I was looking at my FIO cards, I think the reason is that the SLC's need more power to deliver higher performance, they are supposed to deliver 1.5GB/sec but only delivers around 350MB/sec each....
    >
    > Now looking for aux power cables and will retest...
    >
    > Br,
    > Rune
    >
    > -----Original Message-----
    > From: Richard Elling [mailto:richard.elling at richardelling.com]
    > Sent: Friday, October 31, 2014 9:03 AM
    > To: Eric Sproul
    > Cc: Rune Tipsmark; omnios-discuss at lists.omniti.com
    > Subject: Re: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
    >
    >
    > On Oct 31, 2014, at 7:14 AM, Eric Sproul <eric.sproul at circonus.com> wrote:
    >
    >> On Fri, Oct 31, 2014 at 2:33 AM, Rune Tipsmark <rt at steait.net> wrote:
    >>
    >>> Why is this pool showing near 100% busy when the underlying disks
    >>> are doing nothing at all....
    >>
    >> Simply put, it's just how the accounting works in iostat.  It treats
    >> the pool like any other device, so if there is even one outstanding
    >> request to the pool, it counts towards the busy%.  Keith W. from
    >> Joyent explained this recently on the illumos-zfs list:
    >> http://www.listbox.com/member/archive/182191/2014/10/sort/time_rev/pa
    >> g
    >> e/3/entry/18:93/20141017161955:F3E11AB2-563A-11E4-8EDC-D0C677981E2F/
    >>
    >> The TL;DR is: if your pool has more than one disk in it, the
    >> pool-wide busy% is useless.
    >
    > FWIW, we use %busy as an indicator that we can ignore a device/subsystem when looking for performance problems. We don't use it as an indicator of problems. In other words, if the device isn't > 10% busy, forgetabouddit. If it is more busy, look in more detail at the meaningful performance indicators.
    > -- richard
    >
    > _______________________________________________
    > OmniOS-discuss mailing list
    > OmniOS-discuss at lists.omniti.com
    > http://lists.omniti.com/mailman/listinfo/omnios-discuss

- Previous message: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
- Next message: [OmniOS-discuss] zfs pool 100% busy, disks less than 10%
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list