`<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">In my case the SATA disks aren’t on the 1068E.<div class=""><br class=""></div><div class="">-brian</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jan 12, 2016, at 11:19 PM, John Barfield <<a href="mailto:john.barfield@bissinc.com" class="">john.barfield@bissinc.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div id="compose" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; padding-left: 16px; padding-right: 16px; padding-bottom: 8px;" class="" applecontenteditable="true"><div class="">BTW I left off that it has the same LSI controller chipset<br class=""><br class=""><div class="acompli_signature">Sent from<span class="Apple-converted-space"> </span><a href="https://aka.ms/qtex0l" class="">Outlook Mobile</a></div><br class=""></div></div><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">_____________________________<br class="">From: John Barfield <<a href="mailto:john.barfield@bissinc.com" class="">john.barfield@bissinc.com</a>><br class="">Sent: Tuesday, January 12, 2016 10:17 PM<br class="">Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging<br class="">To: <<a href="mailto:discuss@lists.illumos.org" class="">discuss@lists.illumos.org</a>>, omnios-discuss <<a href="mailto:omnios-discuss@lists.omniti.com" class="">omnios-discuss@lists.omniti.com</a>><br class=""><br class=""><br class=""><div class=""><div class="">My input may or may not be valid but Im going to throw it out there anyway :)</div><div class=""><br class=""></div><div class="">do you have any Mpt disconnect errors in /var/adm/messages?<span class="Apple-converted-space"> </span></div><div class=""><br class=""></div><div class="">Also do you have smartmontools installed?<span class="Apple-converted-space"> </span></div><div class=""><br class=""></div><div class="">I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes.</div><div class=""><br class=""></div><div class="">I found the drive that was acting up and pulled it.<span class="Apple-converted-space"> </span></div><div class=""><br class=""></div><div class="">All of a sudden everything miraculously worked amazing.<span class="Apple-converted-space"> </span></div><div class=""><br class=""></div><div class="">I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state.</div><div class=""><br class=""></div><div class="">I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads.<span class="Apple-converted-space"> </span><br class=""><br class="">-barfield<span class="Apple-converted-space"> </span><br class=""><br class=""><div class="x_acompli_signature">Sent from<span class="Apple-converted-space"> </span><a href="https://aka.ms/qtex0l" class="">Outlook Mobile</a></div><br class=""></div><br class=""><br class=""><br class=""><div class="x_gmail_quote">On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger"<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:wonko@4amlunch.net" class="">wonko@4amlunch.net</a>></span><span class="Apple-converted-space"> </span>wrote:<span class="Apple-converted-space"> </span><br class=""><br class=""></div></div><font size="2" class=""><span style="font-size: 10pt;" class=""><div class="PlainText">In the meantime I’ve removed the SLOG and L2ARC just in case. I don’t think that’s it though. At least will have some sort of data point to work with here. :)<span class="Apple-converted-space"> </span><br class=""><br class="">-brian<span class="Apple-converted-space"> </span><br class=""><br class="">> On Jan 12, 2016, at 10:55 PM, Brian Hechinger <<a href="mailto:wonko@4amlunch.net" class="">wonko@4amlunch.net</a>> wrote:<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> Ok, it has happened.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> Checking this here, the pool seems to be fine. I can read and write files.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> except ‘zpool status’ is now currently hanging. I can still read/write from the pool, however.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> I can telnet to port 3260, but restarting target services has hung.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcs -a | grep stmf<span class="Apple-converted-space"> </span><br class="">> online Jan_05 svc:/system/stmf:default<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcs -a | grep target<span class="Apple-converted-space"> </span><br class="">> disabled Jan_05 svc:/system/fcoe_target:default<span class="Apple-converted-space"> </span><br class="">> online Jan_05 svc:/network/iscsi/target:default<span class="Apple-converted-space"> </span><br class="">> online Jan_05 svc:/system/ibsrp/target:default<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcadm restart /system/ibsrp/target<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcadm restart /network/iscsi/target<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcadm restart /system/stmf<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcs -a | grep target<span class="Apple-converted-space"> </span><br class="">> disabled Jan_05 svc:/system/fcoe_target:default<span class="Apple-converted-space"> </span><br class="">> online* 22:43:03 svc:/system/ibsrp/target:default<span class="Apple-converted-space"> </span><br class="">> online* 22:43:13 svc:/network/iscsi/target:default<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share# svcs -a | grep stmf<span class="Apple-converted-space"> </span><br class="">> online* 22:43:18 svc:/system/stmf:default<span class="Apple-converted-space"> </span><br class="">> root@basket1:/tank/Share#<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> I’m doing a crash dump reboot. I’ll post the output somewhere.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> The output of echo '$<threadlist' | mdb -k is attached.<span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">> <threadlist.out><span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class="">>> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik <<a href="mailto:matej@zunaj.si" class="">matej@zunaj.si</a>> wrote:<span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">>> Is the pool usable during comstar hang?<span class="Apple-converted-space"> </span><br class="">>> Can you write and read from the pool (test both, in my case, when pool froze, I wasn’t able to write to the pool, but I could read).<span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">>> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging.<span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">>> Matej<span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">>>> On 08 Jan 2016, at 20:11, Brian Hechinger <<a href="mailto:wonko@4amlunch.net" class="">wonko@4amlunch.net</a>> wrote:<span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> Yeah, I’m using the 1068E to boot from (this has been supported since before Illumos) but that doesn’t have anything accessed by COMSTAR.<span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> It’s the ICH10R SATA that hosts the disks that COMSTAR shares out space from.<span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> -brian<span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel <<a href="mailto:rjahnel@ellipseinc.com" class="">rjahnel@ellipseinc.com</a>> wrote:<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> First off, love SuperMicro good choice IMHO.<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> This board has two on board controllers.<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one)<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> And<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Intel ICH10R SATA (So I'm guessing your using this one.)<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> -----Original Message-----<span class="Apple-converted-space"> </span><br class="">>>>> From: OmniOS-discuss [<span class="Apple-converted-space"> </span><a href="mailto:omnios-discuss-bounces@lists.omniti.com" class="">mailto:omnios-discuss-bounces@lists.omniti.com</a>] On Behalf Of Brian Hechinger<span class="Apple-converted-space"> </span><br class="">>>>> Sent: Friday, January 08, 2016 12:16 PM<span class="Apple-converted-space"> </span><br class="">>>>> To: Matej Zerovnik <<a href="mailto:matej@zunaj.si" class="">matej@zunaj.si</a>><span class="Apple-converted-space"> </span><br class="">>>>> Cc: omnios-discuss <<a href="mailto:omnios-discuss@lists.omniti.com" class="">omnios-discuss@lists.omniti.com</a>><span class="Apple-converted-space"> </span><br class="">>>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Which controller exactly do you have?<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F.<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Do you know firmware version?<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> I’m assuming this is linked to the BIOS version?<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Which hard drives?<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> It might not tell much, but it’s good to have as much information as possible.<span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> When comstar hangs, can you telnet to the iSCSI port?<span class="Apple-converted-space"> </span><br class="">>>>>> What does svcs says, is the service running?<span class="Apple-converted-space"> </span><br class="">>>>>> What happens in you try to restart it?<span class="Apple-converted-space"> </span><br class="">>>>>> How do you restart it?<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> I’ll try all these things next time.<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced).<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> All good info. Thanks!<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> -brian<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Matej<span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> On 08 Jan 2016, at 17:50, Dave Pooser <<a href="mailto:dave-oo@pooserville.com" class="">dave-oo@pooserville.com</a>> wrote:<span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger <<a href="mailto:wonko@4amlunch.net" class="">wonko@4amlunch.net</a>> wrote:<span class="Apple-converted-space"> </span><br class="">>>>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>>>> No, ZFS raid10<span class="Apple-converted-space"> </span><br class="">>>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese?<span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> It's a zpool with multiple mirror vdevs.<span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> --<span class="Apple-converted-space"> </span><br class="">>>>>>> Dave Pooser<span class="Apple-converted-space"> </span><br class="">>>>>>> Cat-Herder-in-Chief, <a href="http://pooserville.com" class="">Pooserville.com</a><span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> _______________________________________________<span class="Apple-converted-space"> </span><br class="">>>>>>> OmniOS-discuss mailing list<span class="Apple-converted-space"> </span><br class="">>>>>>> <a href="mailto:OmniOS-discuss@lists.omniti.com" class="">OmniOS-discuss@lists.omniti.com</a><span class="Apple-converted-space"> </span><br class="">>>>>>><span class="Apple-converted-space"> </span><a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> _______________________________________________<span class="Apple-converted-space"> </span><br class="">>>>>> OmniOS-discuss mailing list<span class="Apple-converted-space"> </span><br class="">>>>>> <a href="mailto:OmniOS-discuss@lists.omniti.com" class="">OmniOS-discuss@lists.omniti.com</a><span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> _______________________________________________<span class="Apple-converted-space"> </span><br class="">>>>> OmniOS-discuss mailing list<span class="Apple-converted-space"> </span><br class="">>>>> <a href="mailto:OmniOS-discuss@lists.omniti.com" class="">OmniOS-discuss@lists.omniti.com</a><span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">><span class="Apple-converted-space"> </span><br class=""><br class=""><br class=""><br class=""><a href="http://www.listbox.com/" class="">http://www.listbox.com</a><span class="Apple-converted-space"> </span><br class=""></div></span></font><div bgcolor="#ffffff" id="listbox-footer" style="width: auto; margin: 0px; padding: 5px; background-color: rgb(255, 255, 255); clear: both; border-top-width: 1px; border-top-style: solid; border-top-color: rgb(204, 204, 204);" class=""><table bgcolor="#ffffff" border="0" cellpadding="0" cellspacing="0" width="100%" style="background-color: rgb(255, 255, 255);" class=""><tbody class=""><tr class=""><td class=""><font color="#333333" size="1" style="font-family: helvetica, sans-serif;" class=""><strong class="">illumos-discuss</strong><span class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a href="https://www.listbox.com/member/archive/182180/=now" title="Go to archives for illumos-discuss" style="text-decoration: none; color: rgb(102, 153, 51); border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(68, 68, 68);" class="">Archives</a><span class="Apple-converted-space"> </span><a border="0" href="https://www.listbox.com/member/archive/rss/182180/26677440-40b316d8" title="RSS feed for illumos-discuss" style="text-decoration: none; color: rgb(102, 153, 51);" class=""><img border="0" src="http://postlink.www.listbox.com/2033704/833487e62783d55fe81f119fb93ef644/26677440/3044d385.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc" class=""></a><span class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a href="https://www.listbox.com/member/?member_id=26677440&id_secret=26677440-8fd7f4fe" title="" style="text-decoration: none; color: rgb(102, 153, 51); border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(68, 68, 68);" class="">Modify</a><span class="Apple-converted-space"> </span>Your Subscription</font></td><td align="right" valign="top" class=""><a href="http://www.listbox.com/" style="border-bottom-style: none;" class=""><img border="0" src="http://postlink.www.listbox.com/2033705/3379085af0f1cf7fc3708f04b4471ae2/26677440/3044d385.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc" title="Powered by Listbox" class=""></a></td></tr></tbody></table></div><br class=""><br class=""></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">OmniOS-discuss mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="mailto:OmniOS-discuss@lists.omniti.com" class="">OmniOS-discuss@lists.omniti.com</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote></div><br class=""></div></body></html>`