# [OmniOS-discuss] How bad are these controller / io errors??

Richard Elling richard.elling at richardelling.com
Wed Aug 21 23:30:56 UTC 2013

- Previous message: [OmniOS-discuss] How bad are these controller / io errors??
- Next message: [OmniOS-discuss] How bad are these controller / io errors??
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    On Aug 21, 2013, at 3:53 PM, steve at linuxsuite.org wrote:

    >> SATA drives behind SAS expanders have a pathological error case.  If a
    >> drive encounters errors and it needs to be reset then the entire set of
    >> drives will take the reset.  If any io is in flight this will error as a
    >> result often causing another reset.  The result can best be described as a
    >> cascade failure.
    >>
    >> There may be things software can do to mitigate this but experience is
    >> that it doesn't.
    >
    >        Hmmm..... I have put.
    >
    >            allow-bus-device-reset=0;
    >
    >      In sd.conf
    >
    >      as it seemed to mitigate that problem?

    IIRC, this is a nop for mpt and mptsas drivers.
     -- richard

    >
    >
    >>
    >> So just because the array is working fine now does not mean that it won't
    >> fail tragically when the first problems occur.
    >>
    >> I have personal experience with this failure mode.
    >>
    >> As a result I strongly discourage the use of SATA drives unless they are
    >> directly connected to the hba without any expanders or port multipliers.
    >>
    >> You ignore this advice at your own risk.  Don't penny pinch on the drives
    >> It ALWAYS costs you in the long run.
    >>
    >
    >         The advice in this thread, wrt SATA vs SAS, has been my thoughts
    > and position
    > for a while based on my initial research. I was required to explore and
    > verify this option
    > simply because of the significant economic  benefit if it was possible.
    > Thanks for your
    > input. I will use this to make my case
    >
    >    So now I just need some cheap SAS drives ;-)
    >
    >    thanx - steve
    >
    >> Sent from my iPhone
    >>
    >> On Aug 19, 2013, at 4:25 PM, steve at linuxsuite.org wrote:
    >>
    >>>> Can't agree more here. Desktop firmwares are designed to try harder to
    >>>> never return an error.  While this is what home users want, it's an
    >>>> anathema to large configurations with redundancy where you would prefer
    >>>> to
    >>>> just get the error so you can handle it - usually by doing the op on
    >>>> another drive.
    >>>>
    >>>> Use enterprise drives if you love your data and your ability to access
    >>>> it.
    >>>
    >>>       Yes I know, I know....  Desktop hardware sucks. Didn't know there
    >>> was
    >>> Enterprise (ie none Desktop) SATA. Perhaps the solution is to  partition
    >>> the data and do "deep" storage that never gets read on large bulk SATA
    >>> and
    >>> then have a smaller SAS pool for the often read/written stuff.
    >>>
    >>>     Anyway, large storage rollouts will always be looking for
    >>> the cheapest solution. Even if it isn't perfect. I do backups
    >>> to a SATA pool (behind SAS expanders) with zfs send,.. no problems
    >>> errors
    >>> whatever YET!!
    >>> (cross fingers)
    >>>
    >>>      -steve
    >>>
    >>>>
    >>>> Sent from my iPhone
    >>>>
    >>>> On Aug 19, 2013, at 2:47 PM, Eric Sproul <esproul at omniti.com> wrote:
    >>>>
    >>>>> On Mon, Aug 19, 2013 at 5:35 PM,  <steve at linuxsuite.org> wrote:
    >>>>>>   4T SATA  here
    >>>>>>
    >>>>>> http://www.newegg.com/Product/Product.aspx?Item=N82E16822178338
    >>>>>>
    >>>>>>        are $179
    >>>>>
    >>>>> That's a desktop drive, not nearline enterprise, so it's
    >>>>> apples-oranges.  You get what you pay for.  Desktop drives can take
    >>>>> much, much longer to respond to commands, leading the HBA/expander to
    >>>>> declare them dead and reset them, and the rest is history.
    >>>>>
    >>>>> Avoid desktop parts.  :)
    >>>>>
    >>>>> Eric
    >>>>> _______________________________________________
    >>>>> OmniOS-discuss mailing list
    >>>>> OmniOS-discuss at lists.omniti.com
    >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
    >>>
    >>> _______________________________________________
    >>> OmniOS-discuss mailing list
    >>> OmniOS-discuss at lists.omniti.com
    >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
    >>
    >
    >
    > _______________________________________________
    > OmniOS-discuss mailing list
    > OmniOS-discuss at lists.omniti.com
    > http://lists.omniti.com/mailman/listinfo/omnios-discuss

    --

    Richard.Elling at RichardElling.com
    +1-760-896-4422

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20130821/e5c2918f/attachment.html>

- Previous message: [OmniOS-discuss] How bad are these controller / io errors??
- Next message: [OmniOS-discuss] How bad are these controller / io errors??
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list