From rt at steait.net  Mon Feb  2 00:21:46 2015
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 2 Feb 2015 00:21:46 +0000
Subject: [OmniOS-discuss] Windows crashes my ZFS box
Message-ID: <1422836506599.46682@steait.net>

hi all,



I got some major problems... when using Windows and Fibre Channel I am able to kill my ZFS box totally for at least 15 minutes... it simply drops all connections to all hosts connected via FC. This happens under load, for example doing backups writing to the ZFS, running IO Meter against my ZFS...



I have looked at queue depth/length and other stuff, I just cannot seem to find out how this happens... I have tested on 3 different Windows machines and 3 difference ZFS boxes - I have ESXi Servers connected to these ZFS boxes as well and no problem there no matter how much load I put on the LUN's.



I tried sync=always, sync=disabled, with and without log devices, dedup=on, dedup=off, volume based lun, thin provisioned lun, zfs based thin provisioned lun... you name it, I tried it...



tried all kinds of queue lengths from Windows, default is 65535, tried 16, 32, 64, 256 etc... if I put enough stress on the lun presented to Windows it will cause the ZFS box to drop all FC connections for up to 15 minutes.. a reboot is not possible as it will hang for the same amount of minutes... might as well wait for it to come back...



Latest FW on all items, HBA, Switch etc. Monitoring shows a distributed load on the ports as expected using Round Robin and MPIO.



One thing that irritates me is that I don't get any more than ~80-120 MB/sec (sync=always) throughput when writing to this LUN in Windows, where I get 6-700 MB/sec (sync=always) when writing from a VM on ESXi... The abysmal performance is a pain, but the fact that I can downright crash or hang my ZFS box just by running IOMeter is disturbing...



Any ideas why this might happen? Seems to me like a queue problem but I can't really get any closer than that... maybe Windows is just crappy at handling Fibre Channel... however no problems against HP EVA Storage.... same machine, same tests....



br,

Rune




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150202/d59cf1f3/attachment-0001.html>

From chip at innovates.com  Mon Feb  2 14:50:42 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Mon, 2 Feb 2015 08:50:42 -0600
Subject: [OmniOS-discuss] Windows crashes my ZFS box
In-Reply-To: <1422836506599.46682@steait.net>
References: <1422836506599.46682@steait.net>
Message-ID: <CALeZrrSyrWexbejHxo-3G4grLvZcEX9xB4bzUHg927Ne45V0VQ@mail.gmail.com>

On Sun, Feb 1, 2015 at 6:21 PM, Rune Tipsmark <rt at steait.net> wrote:

> I got some major problems... when using Windows and Fibre Channel I am
> able to kill my ZFS box totally for at least 15 minutes... it simply drops
> all connections to all hosts connected via FC. This happens under load, for
> example doing backups writing to the ZFS, running IO Meter against my ZFS...
>

...

>
> Latest FW on all items, HBA, Switch etc. Monitoring shows a distributed
> load on the ports as expected using Round Robin and MPIO.
>
> This might be a shot in the dark, but the latest firmware on LSI HBAs is
known to have serious problems.  It has more to do with data corrupting, so
I'm not sure this is your cause.  Use P18 or P19, but not P20.

-Chip


>
>
> One thing that irritates me is that I don't get any more than ~80-120
> MB/sec (sync=always) throughput when writing to this LUN in Windows, where
> I get 6-700 MB/sec (sync=always) when writing from a VM on ESXi... The
> abysmal performance is a pain, but the fact that I can downright crash or
> hang my ZFS box just by running IOMeter is disturbing...
>
>
>
> Any ideas why this might happen? Seems to me like a queue problem but I
> can't really get any closer than that... maybe Windows is just crappy at
> handling Fibre Channel... however no problems against HP EVA Storage....
> same machine, same tests....
>
>
>
> br,
>
> Rune
>
>
>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150202/c8700902/attachment.html>

From johan.kragsterman at capvert.se  Mon Feb  2 15:18:18 2015
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Mon, 2 Feb 2015 16:18:18 +0100
Subject: [OmniOS-discuss] Ang: Re:  Windows crashes my ZFS box
In-Reply-To: <CALeZrrSyrWexbejHxo-3G4grLvZcEX9xB4bzUHg927Ne45V0VQ@mail.gmail.com>
References: <CALeZrrSyrWexbejHxo-3G4grLvZcEX9xB4bzUHg927Ne45V0VQ@mail.gmail.com>,
	<1422836506599.46682@steait.net>
Message-ID: <OF5ED0551C.D089EFDB-ONC1257DE0.00533261-C1257DE0.005412B0@inse.com>


Hi!

-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Rune Tipsmark <rt at steait.net>
Fr?n: "Schweiss, Chip" 
S?nt av: "OmniOS-discuss" 
Datum: 2015-02-02 15:52
Kopia: "omnios-discuss at lists.omniti.com" <omnios-discuss at lists.omniti.com>
?rende: Re: [OmniOS-discuss] Windows crashes my ZFS box

On Sun, Feb 1, 2015 at 6:21 PM, Rune Tipsmark <rt at steait.net> wrote:
I got some major problems... when using Windows and Fibre Channel I am able to kill my ZFS box totally for at least 15 minutes... it simply drops all connections to all hosts connected via FC. This happens under load, for example doing backups writing to the ZFS, running IO Meter against my ZFS...

... 

Latest FW on all items, HBA, Switch etc. Monitoring shows a distributed load on the ports as expected using Round Robin and MPIO.



This might be a shot in the dark, but the latest firmware on LSI HBAs is known to have serious problems.? It has more to do with data corrupting, so I'm not sure this is your cause.? Use P18 or P19, but not P20.

-Chip




Can't be that, cos' it works fine with other host operating systems.

No, when I think about it, the only thing I can imagine is the windows handling of sparse volumes. Are your LU's based on sparse volumes? I had a problem a couple of yrs ago with oVirt nodes, that is the oVirt virtualization management system hypervisor nodes. When I wanted to install these to the SAN, I got problems due to nested qcow volumes. I didn't really dig into it back then, but I know that the problems was the implementation of LVM/qcow they used, on top of ZFS/COMSTAR sparse volumes/LU's.

So I thought it MIGHT be something similar on window$, like the software volume handling or so. I am NO expert on win, but it might be worth trying to configure a non sparse volume for a LU to send to window$?

Johan






?

One thing that irritates me is that I don't get any more than ~80-120 MB/sec (sync=always)?throughput when writing to this LUN in Windows, where I get 6-700 MB/sec (sync=always)?when writing from a VM on ESXi... The abysmal performance is a pain, but the fact that I can downright crash or hang my ZFS box just by running IOMeter is disturbing...

?

Any ideas why this might happen? Seems to me like a queue problem but I can't really get any closer than that... maybe Windows is just crappy at handling Fibre Channel... however no problems against HP EVA Storage.... same machine, same tests....

?

br,

Rune

?

?


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss


From danmcd at omniti.com  Mon Feb  2 20:39:19 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 2 Feb 2015 15:39:19 -0500
Subject: [OmniOS-discuss] Update to r151006 --> 088
Message-ID: <B2A1FB84-4238-48FC-9F42-3E470475DFEA@omniti.com>

A few customers required some backports, so r151006 got one last update.  Here are the release notes:

http://omnios.omniti.com/wiki.php/ReleaseNotes/r151006#r151006_088

The bits have been in the repo servers themselves for a couple of weeks, but now there's accompanying release media.  Check either your openssl or mpt_sas versions to confirm things.  Just typing "openssl version" for the former, and "pkg list -v mpt_sas" should now show:

pkg://omnios/driver/storage/mpt_sas at 0.5.11,5.11-0.151006:20150119T182628Z    i--

Thanks!
Dan


From sergey57 at gmail.com  Tue Feb  3 18:06:58 2015
From: sergey57 at gmail.com (sergey ivanov)
Date: Tue, 3 Feb 2015 13:06:58 -0500
Subject: [OmniOS-discuss] A few packages
Message-ID: <CAAv=BDHBQFdY_0qWeLEHL34+-6fhB8dG385V4tZLJ2Bd2w9jfA@mail.gmail.com>

Hi,
I've published today in
https://github.com/seriv/omnios-build/tree/cs.umd.edu a branch forked
from template with our build scripts for dovecot IMAP server with
pigeonhole Managesieve server and with compiled as a plugin ldap
authentication (linked against omniti-ms' openldap-client package).
Also updated versions of other packages there (mc, fail2ban...).

-- 
Regards,
Sergey Ivanov | sergey57 at gmail.com
http://www.linkedin.com/pub/sergey-ivanov/8/270/a09

From danmcd at omniti.com  Tue Feb  3 18:37:03 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 13:37:03 -0500
Subject: [OmniOS-discuss] A few packages
In-Reply-To: <CAAv=BDHBQFdY_0qWeLEHL34+-6fhB8dG385V4tZLJ2Bd2w9jfA@mail.gmail.com>
References: <CAAv=BDHBQFdY_0qWeLEHL34+-6fhB8dG385V4tZLJ2Bd2w9jfA@mail.gmail.com>
Message-ID: <31403E7E-F432-4CC4-9518-E404F410B19C@omniti.com>


> On Feb 3, 2015, at 1:06 PM, sergey ivanov <sergey57 at gmail.com> wrote:
> 
> Hi,
> I've published today in
> https://github.com/seriv/omnios-build/tree/cs.umd.edu a branch forked
> from template with our build scripts for dovecot IMAP server with
> pigeonhole Managesieve server and with compiled as a plugin ldap
> authentication (linked against omniti-ms' openldap-client package).
> Also updated versions of other packages there (mc, fail2ban...).

And netperf.  Thank you! That's one I may put into master (maybe after 014 ships, maybe even FOR 014...).

Dan


From danmcd at omniti.com  Tue Feb  3 20:35:30 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 15:35:30 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
Message-ID: <E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>

There is no TL;DR for this.  Please read all of this before you upgrade.

This is a big one.  I've updated install media AND the repo servers. Users who use "pkg update" will be getting a big wad (I'm not taking chances with missed packages or components).  I'm sorry it took longer than I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed before I released the next bloody update.

This is r151013-20150203:

- omnios-build master branch, revision 0c38601

- NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own section below.

- OpenSSL is now at 1.0.2.

- One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we can't catch up until VND is upstreamed from SmartOS).

- Longstanding bug with GCC 4.4.4's "cc1" not being properly linked. (Thanks Richard Yao of ZFS-on-Linux for catching this.)

- Curl is now at 7.40.0.

- NTP dependency fix, from the community. (Thanks to "takashiary".)

- Git is now at 2.2.1.

- ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri "lotheac" Tirkkonen.)

- illumos-omnios master branch, revision aef6850 (last illumos-gate merge 6309835)

- Several ZFS enhancements, and sometimes followup fixes for them (one, 5531, was what made this release take longer than it should have).

- dis(1) supports cross-target disassembly  (Illumos 3317)

- NFS improvements in the authentication cache (Illumos 5509)

- Several ELF safety improvements from Rich Lowe.

- kmem_reap() performance improvements

- preadv() and pwritev()
 NOTE: KVM's poor handling of these (assuming all the world's Linux) is why we have a cherrypick above.

- illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in omnios-userland, that will be updated closer to 014's release).

- Developer prototypes updated to 2015

- Introducing the linked-ipkg ("lipkg") zone brand.  See below about how this relates to the new pkg(5) changes.

- Reducing RAM used in managing ZFS cache devices

- Many new, corrected, and updated man pages.

- Longstanding mailx(1) overflow fixes now in place.


NEW FOR 2015 -- updated pkg(5)

You may have known this from previous bloody releases, but we've been using the same oldish version of pkg(5) since at least r151006 in OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks things like "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old r151006 version of pkg(5).  That changes now.

If you're interested in source, our new repo for pkg(5) is http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.  It's a downstream of OI Hipster's for now.

The OpenIndiana Hipster folks have done some work to bring pkg(5) more up to date with its upstream. The new pkg(5) contains many under-the-hood improvements. It also includes two user-visible improvements:

	- A bit more visible status information during pkg(1) operations.

	- The ability to link images.

Linked images is the big one.  Way upstream, linked images are the new default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI Hipster, linked-images are currently disabled.  We decided to take a middle ground:


THE LINKED IPS BRAND (lipkg)

Currently deployed ipkg-branded zones stay the same as they have in previous OmniOS releases.  You update them how you're used to updating them:  Either the documented OmniOS way, or "Dan's way", where you mount the newly upgraded BE and then update each zone prior to reboot.

If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach) you may change its brand the the new linked-ipkg (lipkg) brand.  The default for a new zone is still "ipkg", so if you want lipkg, you must explicitly "set brand=lipkg" in the zonecfg interaction or script.

How to convert an existing "ipkg" zone to an "lipkg" zone:

1.) Install the "lipkg" brand:  "pkg install lipkg".

2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone> detach"

3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".

4.) Attach the zone with the -u flag. pkg(5) needs to run its update check regardless:  "zoneadm -z <zone> attach -u".

5.) Boot the zone: "zoneadm -z <zone> boot".

The zone <zone>'s image will be linked to the global zone's image.

Once you are running with lipkg zones, new things happen upon subsequent "pkg update"s.  Any packages in the global zone that get updated will also be updated in the lipkg zones at the same time.  This includes more than just the "omnios" publisher as well, which is why lipkg zones aren't for everyone.  If, like me, you tend to currently use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg update" where all zones are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new BE is created, and all of the zones in the new BE are already updated.  Furthermore, if you want to have the semantics of not losing log data during the upgrade, you now merely need to halt the zones, then "pkg update", and reboot.  If your zones are autoboot (which they should be) everything's up and running with shiny new bits.

Attached as a .txt file is a sample transcript of a linked-image upgrade with a global zone and a non-global zone.

Please give this one a try, and we'd really appreciate results from:

	* lipkg-branded zones.

	* Kayak images.

	* The ISO installation experience.

	* ZFS stressing (there may be some residual issues here from upstream, so be warned).

We will really appreciate the feedback.

Thanks,
Dan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: transcript.txt
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/0ff95f1e/attachment-0001.txt>
-------------- next part --------------



From jesus at omniti.com  Tue Feb  3 20:51:49 2015
From: jesus at omniti.com (Theo Schlossnagle)
Date: Tue, 3 Feb 2015 15:51:49 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
Message-ID: <CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>

Fantastic work Dan!  We're all looking forward to 014!

On Tue, Feb 3, 2015 at 3:35 PM, Dan McDonald <danmcd at omniti.com> wrote:

> There is no TL;DR for this.  Please read all of this before you upgrade.
>
> This is a big one.  I've updated install media AND the repo servers. Users
> who use "pkg update" will be getting a big wad (I'm not taking chances with
> missed packages or components).  I'm sorry it took longer than I expected,
> but there was an illumos-gate ZFS bug (5531) I wanted fixed before I
> released the next bloody update.
>
> This is r151013-20150203:
>
> - omnios-build master branch, revision 0c38601
>
> - NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own
> section below.
>
> - OpenSSL is now at 1.0.2.
>
> - One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we
> can't catch up until VND is upstreamed from SmartOS).
>
> - Longstanding bug with GCC 4.4.4's "cc1" not being properly linked.
> (Thanks Richard Yao of ZFS-on-Linux for catching this.)
>
> - Curl is now at 7.40.0.
>
> - NTP dependency fix, from the community. (Thanks to "takashiary".)
>
> - Git is now at 2.2.1.
>
> - ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri
> "lotheac" Tirkkonen.)
>
> - illumos-omnios master branch, revision aef6850 (last illumos-gate merge
> 6309835)
>
> - Several ZFS enhancements, and sometimes followup fixes for them (one,
> 5531, was what made this release take longer than it should have).
>
> - dis(1) supports cross-target disassembly  (Illumos 3317)
>
> - NFS improvements in the authentication cache (Illumos 5509)
>
> - Several ELF safety improvements from Rich Lowe.
>
> - kmem_reap() performance improvements
>
> - preadv() and pwritev()
>  NOTE: KVM's poor handling of these (assuming all the world's Linux) is
> why we have a cherrypick above.
>
> - illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in
> omnios-userland, that will be updated closer to 014's release).
>
> - Developer prototypes updated to 2015
>
> - Introducing the linked-ipkg ("lipkg") zone brand.  See below about how
> this relates to the new pkg(5) changes.
>
> - Reducing RAM used in managing ZFS cache devices
>
> - Many new, corrected, and updated man pages.
>
> - Longstanding mailx(1) overflow fixes now in place.
>
>
> NEW FOR 2015 -- updated pkg(5)
>
> You may have known this from previous bloody releases, but we've been
> using the same oldish version of pkg(5) since at least r151006 in OmniOS.
> Bloody releases have had a half-finished pkg(5) that breaks things like
> "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old
> r151006 version of pkg(5).  That changes now.
>
> If you're interested in source, our new repo for pkg(5) is
> http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.
> It's a downstream of OI Hipster's for now.
>
> The OpenIndiana Hipster folks have done some work to bring pkg(5) more up
> to date with its upstream. The new pkg(5) contains many under-the-hood
> improvements. It also includes two user-visible improvements:
>
>         - A bit more visible status information during pkg(1) operations.
>
>         - The ability to link images.
>
> Linked images is the big one.  Way upstream, linked images are the new
> default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI
> Hipster, linked-images are currently disabled.  We decided to take a middle
> ground:
>
>
> THE LINKED IPS BRAND (lipkg)
>
> Currently deployed ipkg-branded zones stay the same as they have in
> previous OmniOS releases.  You update them how you're used to updating
> them:  Either the documented OmniOS way, or "Dan's way", where you mount
> the newly upgraded BE and then update each zone prior to reboot.
>
> If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach)
> you may change its brand the the new linked-ipkg (lipkg) brand.  The
> default for a new zone is still "ipkg", so if you want lipkg, you must
> explicitly "set brand=lipkg" in the zonecfg interaction or script.
>
> How to convert an existing "ipkg" zone to an "lipkg" zone:
>
> 1.) Install the "lipkg" brand:  "pkg install lipkg".
>
> 2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone>
> detach"
>
> 3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".
>
> 4.) Attach the zone with the -u flag. pkg(5) needs to run its update check
> regardless:  "zoneadm -z <zone> attach -u".
>
> 5.) Boot the zone: "zoneadm -z <zone> boot".
>
> The zone <zone>'s image will be linked to the global zone's image.
>
> Once you are running with lipkg zones, new things happen upon subsequent
> "pkg update"s.  Any packages in the global zone that get updated will also
> be updated in the lipkg zones at the same time.  This includes more than
> just the "omnios" publisher as well, which is why lipkg zones aren't for
> everyone.  If, like me, you tend to currently use the "Dan's way" of
> upgrading, lipkg zones will save you time.  "pkg update" where all zones
> are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new
> BE is created, and all of the zones in the new BE are already updated.
> Furthermore, if you want to have the semantics of not losing log data
> during the upgrade, you now merely need to halt the zones, then "pkg
> update", and reboot.  If your zones are autoboot (which they should be)
> everything's up and running with shiny new bits.
>
> Attached as a .txt file is a sample transcript of a linked-image upgrade
> with a global zone and a non-global zone.
>
> Please give this one a try, and we'd really appreciate results from:
>
>         * lipkg-branded zones.
>
>         * Kayak images.
>
>         * The ISO installation experience.
>
>         * ZFS stressing (there may be some residual issues here from
> upstream, so be warned).
>
> We will really appreciate the feedback.
>
> Thanks,
> Dan
>
>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>


-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/628dbd29/attachment.html>

From chip at innovates.com  Tue Feb  3 20:56:28 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Tue, 3 Feb 2015 14:56:28 -0600
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
Message-ID: <CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>

Excellent!

I'll put it on my current project system and get you some feedback.

Of particular interest, were there any mpt_sas patches in there?

Cheers!
-Chip

On Tue, Feb 3, 2015 at 2:51 PM, Theo Schlossnagle <jesus at omniti.com> wrote:

> Fantastic work Dan!  We're all looking forward to 014!
>
> On Tue, Feb 3, 2015 at 3:35 PM, Dan McDonald <danmcd at omniti.com> wrote:
>
>> There is no TL;DR for this.  Please read all of this before you upgrade.
>>
>> This is a big one.  I've updated install media AND the repo servers.
>> Users who use "pkg update" will be getting a big wad (I'm not taking
>> chances with missed packages or components).  I'm sorry it took longer than
>> I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed
>> before I released the next bloody update.
>>
>> This is r151013-20150203:
>>
>> - omnios-build master branch, revision 0c38601
>>
>> - NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own
>> section below.
>>
>> - OpenSSL is now at 1.0.2.
>>
>> - One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because
>> we can't catch up until VND is upstreamed from SmartOS).
>>
>> - Longstanding bug with GCC 4.4.4's "cc1" not being properly linked.
>> (Thanks Richard Yao of ZFS-on-Linux for catching this.)
>>
>> - Curl is now at 7.40.0.
>>
>> - NTP dependency fix, from the community. (Thanks to "takashiary".)
>>
>> - Git is now at 2.2.1.
>>
>> - ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to
>> Lauri "lotheac" Tirkkonen.)
>>
>> - illumos-omnios master branch, revision aef6850 (last illumos-gate merge
>> 6309835)
>>
>> - Several ZFS enhancements, and sometimes followup fixes for them (one,
>> 5531, was what made this release take longer than it should have).
>>
>> - dis(1) supports cross-target disassembly  (Illumos 3317)
>>
>> - NFS improvements in the authentication cache (Illumos 5509)
>>
>> - Several ELF safety improvements from Rich Lowe.
>>
>> - kmem_reap() performance improvements
>>
>> - preadv() and pwritev()
>>  NOTE: KVM's poor handling of these (assuming all the world's Linux) is
>> why we have a cherrypick above.
>>
>> - illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in
>> omnios-userland, that will be updated closer to 014's release).
>>
>> - Developer prototypes updated to 2015
>>
>> - Introducing the linked-ipkg ("lipkg") zone brand.  See below about how
>> this relates to the new pkg(5) changes.
>>
>> - Reducing RAM used in managing ZFS cache devices
>>
>> - Many new, corrected, and updated man pages.
>>
>> - Longstanding mailx(1) overflow fixes now in place.
>>
>>
>> NEW FOR 2015 -- updated pkg(5)
>>
>> You may have known this from previous bloody releases, but we've been
>> using the same oldish version of pkg(5) since at least r151006 in OmniOS.
>> Bloody releases have had a half-finished pkg(5) that breaks things like
>> "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old
>> r151006 version of pkg(5).  That changes now.
>>
>> If you're interested in source, our new repo for pkg(5) is
>> http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.
>> It's a downstream of OI Hipster's for now.
>>
>> The OpenIndiana Hipster folks have done some work to bring pkg(5) more up
>> to date with its upstream. The new pkg(5) contains many under-the-hood
>> improvements. It also includes two user-visible improvements:
>>
>>         - A bit more visible status information during pkg(1) operations.
>>
>>         - The ability to link images.
>>
>> Linked images is the big one.  Way upstream, linked images are the new
>> default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI
>> Hipster, linked-images are currently disabled.  We decided to take a middle
>> ground:
>>
>>
>> THE LINKED IPS BRAND (lipkg)
>>
>> Currently deployed ipkg-branded zones stay the same as they have in
>> previous OmniOS releases.  You update them how you're used to updating
>> them:  Either the documented OmniOS way, or "Dan's way", where you mount
>> the newly upgraded BE and then update each zone prior to reboot.
>>
>> If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone>
>> detach) you may change its brand the the new linked-ipkg (lipkg) brand.
>> The default for a new zone is still "ipkg", so if you want lipkg, you must
>> explicitly "set brand=lipkg" in the zonecfg interaction or script.
>>
>> How to convert an existing "ipkg" zone to an "lipkg" zone:
>>
>> 1.) Install the "lipkg" brand:  "pkg install lipkg".
>>
>> 2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z
>> <zone> detach"
>>
>> 3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".
>>
>> 4.) Attach the zone with the -u flag. pkg(5) needs to run its update
>> check regardless:  "zoneadm -z <zone> attach -u".
>>
>> 5.) Boot the zone: "zoneadm -z <zone> boot".
>>
>> The zone <zone>'s image will be linked to the global zone's image.
>>
>> Once you are running with lipkg zones, new things happen upon subsequent
>> "pkg update"s.  Any packages in the global zone that get updated will also
>> be updated in the lipkg zones at the same time.  This includes more than
>> just the "omnios" publisher as well, which is why lipkg zones aren't for
>> everyone.  If, like me, you tend to currently use the "Dan's way" of
>> upgrading, lipkg zones will save you time.  "pkg update" where all zones
>> are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new
>> BE is created, and all of the zones in the new BE are already updated.
>> Furthermore, if you want to have the semantics of not losing log data
>> during the upgrade, you now merely need to halt the zones, then "pkg
>> update", and reboot.  If your zones are autoboot (which they should be)
>> everything's up and running with shiny new bits.
>>
>> Attached as a .txt file is a sample transcript of a linked-image upgrade
>> with a global zone and a non-global zone.
>>
>> Please give this one a try, and we'd really appreciate results from:
>>
>>         * lipkg-branded zones.
>>
>>         * Kayak images.
>>
>>         * The ISO installation experience.
>>
>>         * ZFS stressing (there may be some residual issues here from
>> upstream, so be warned).
>>
>> We will really appreciate the feedback.
>>
>> Thanks,
>> Dan
>>
>>
>>
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>
>
> --
>
> Theo Schlossnagle
>
> http://omniti.com/is/theo-schlossnagle
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/e72e1b53/attachment-0001.html>

From danmcd at omniti.com  Tue Feb  3 20:59:09 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 15:59:09 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
Message-ID: <D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>


> On Feb 3, 2015, at 3:56 PM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> Excellent!
> 
> I'll put it on my current project system and get you some feedback.
> 
> Of particular interest, were there any mpt_sas patches in there?

It has the latest illumos-gate versions of things as of two days ago.  Very recently (mid-January) we backported all mpt_sas stuff into 012 and 006.  So if you're updated to the latest 012, you won't have much new in mpt_sas.  ZFS, OTOH, has newness in it, and alas, not all of it is good (5531 has siblings, apparently).

Dan


From chip at innovates.com  Tue Feb  3 21:07:19 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Tue, 3 Feb 2015 15:07:19 -0600
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
	<D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
Message-ID: <CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>

On Tue, Feb 3, 2015 at 2:59 PM, Dan McDonald <danmcd at omniti.com> wrote:

>
> It has the latest illumos-gate versions of things as of two days ago.
> Very recently (mid-January) we backported all mpt_sas stuff into 012 and
> 006.  So if you're updated to the latest 012, you won't have much new in
> mpt_sas.  ZFS, OTOH, has newness in it, and alas, not all of it is good
> (5531 has siblings, apparently).
>
> Dan
>
>
Good to know.  Regardless, I will run bloody until I'm ready to go live on
this system.   Hopefully I can contribute some valuable information, even
if it is in the form of crash dumps.

-Chip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/df57b5ad/attachment.html>

From danmcd at omniti.com  Tue Feb  3 21:08:18 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 16:08:18 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
	<D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
	<CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>
Message-ID: <B15C3F78-7148-4FFE-B14F-70A2B5CC0DB1@omniti.com>


> On Feb 3, 2015, at 4:07 PM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> Good to know.  Regardless, I will run bloody until I'm ready to go live on this system.   Hopefully I can contribute some valuable information, even if it is in the form of crash dumps.   

Crash dumps are appreciated.  Also, you saw what else (packaging) I'm interested in for this particular one.

Thanks!
Dan


From info at houseofancients.nl  Tue Feb  3 22:38:56 2015
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Tue, 3 Feb 2015 22:38:56 +0000
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED48886CF466@vEX01.mindstorm-internet.local>

Hi Dan,

Immediately after updating, drivers for ntxn ethernet are gone ( they were manually installed, as I had "funky stuff" with the IGB nics), and need reinstalling.
Is that normal behavior?
Rolling back to the before snapshot, and drivers are there again, and traffic is back to normal

Met vriendelijke groet / With kind regards,



               Floris van Essen

Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Dan McDonald
Verzonden: dinsdag 3 februari 2015 21:36
Aan: <omnios-discuss at lists.omniti.com>
Onderwerp: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)

There is no TL;DR for this.  Please read all of this before you upgrade.

This is a big one.  I've updated install media AND the repo servers. Users who use "pkg update" will be getting a big wad (I'm not taking chances with missed packages or components).  I'm sorry it took longer than I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed before I released the next bloody update.

This is r151013-20150203:

- omnios-build master branch, revision 0c38601

- NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own section below.

- OpenSSL is now at 1.0.2.

- One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we can't catch up until VND is upstreamed from SmartOS).

- Longstanding bug with GCC 4.4.4's "cc1" not being properly linked. (Thanks Richard Yao of ZFS-on-Linux for catching this.)

- Curl is now at 7.40.0.

- NTP dependency fix, from the community. (Thanks to "takashiary".)

- Git is now at 2.2.1.

- ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri "lotheac" Tirkkonen.)

- illumos-omnios master branch, revision aef6850 (last illumos-gate merge 6309835)

- Several ZFS enhancements, and sometimes followup fixes for them (one, 5531, was what made this release take longer than it should have).

- dis(1) supports cross-target disassembly  (Illumos 3317)

- NFS improvements in the authentication cache (Illumos 5509)

- Several ELF safety improvements from Rich Lowe.

- kmem_reap() performance improvements

- preadv() and pwritev()
 NOTE: KVM's poor handling of these (assuming all the world's Linux) is why we have a cherrypick above.

- illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in omnios-userland, that will be updated closer to 014's release).

- Developer prototypes updated to 2015

- Introducing the linked-ipkg ("lipkg") zone brand.  See below about how this relates to the new pkg(5) changes.

- Reducing RAM used in managing ZFS cache devices

- Many new, corrected, and updated man pages.

- Longstanding mailx(1) overflow fixes now in place.


NEW FOR 2015 -- updated pkg(5)

You may have known this from previous bloody releases, but we've been using the same oldish version of pkg(5) since at least r151006 in OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks things like "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old r151006 version of pkg(5).  That changes now.

If you're interested in source, our new repo for pkg(5) is http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.  It's a downstream of OI Hipster's for now.

The OpenIndiana Hipster folks have done some work to bring pkg(5) more up to date with its upstream. The new pkg(5) contains many under-the-hood improvements. It also includes two user-visible improvements:

        - A bit more visible status information during pkg(1) operations.

        - The ability to link images.

Linked images is the big one.  Way upstream, linked images are the new default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI Hipster, linked-images are currently disabled.  We decided to take a middle ground:


THE LINKED IPS BRAND (lipkg)

Currently deployed ipkg-branded zones stay the same as they have in previous OmniOS releases.  You update them how you're used to updating them:  Either the documented OmniOS way, or "Dan's way", where you mount the newly upgraded BE and then update each zone prior to reboot.

If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach) you may change its brand the the new linked-ipkg (lipkg) brand.  The default for a new zone is still "ipkg", so if you want lipkg, you must explicitly "set brand=lipkg" in the zonecfg interaction or script.

How to convert an existing "ipkg" zone to an "lipkg" zone:

1.) Install the "lipkg" brand:  "pkg install lipkg".

2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone> detach"

3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".

4.) Attach the zone with the -u flag. pkg(5) needs to run its update check regardless:  "zoneadm -z <zone> attach -u".

5.) Boot the zone: "zoneadm -z <zone> boot".

The zone <zone>'s image will be linked to the global zone's image.

Once you are running with lipkg zones, new things happen upon subsequent "pkg update"s.  Any packages in the global zone that get updated will also be updated in the lipkg zones at the same time.  This includes more than just the "omnios" publisher as well, which is why lipkg zones aren't for everyone.  If, like me, you tend to currently use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg update" where all zones are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new BE is created, and all of the zones in the new BE are already updated.  Furthermore, if you want to have the semantics of not losing log data during the upgrade, you now merely need to halt the zones, then "pkg update", and reboot.  If your zones are autoboot (which they should be) everything's up and running with shiny new bits.

Attached as a .txt file is a sample transcript of a linked-image upgrade with a global zone and a non-global zone.

Please give this one a try, and we'd really appreciate results from:

        * lipkg-branded zones.

        * Kayak images.

        * The ISO installation experience.

        * ZFS stressing (there may be some residual issues here from upstream, so be warned).

We will really appreciate the feedback.

Thanks,
Dan

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/d80b41c6/attachment-0001.html>

From tobi at oetiker.ch  Tue Feb  3 22:46:38 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Tue, 3 Feb 2015 23:46:38 +0100 (CET)
Subject: [OmniOS-discuss] in.tftpd not working and how to fix it
Message-ID: <alpine.DEB.2.10.1502032339230.14595@froburg.oetiker.ch>

caveat: due to the kvm performance problem I am still running 010
... but maybe this is also a problem on more recent versions ...

so here goes:

after upgrading to 010 the tftp service on our omnios box stopped
working properly ... it served a few bytes of the request and then
timed out ...

we investigated for a bit and then moved our tftp service to linux.

Today I ran into the problem again and found the solution:

on http://omnios.omniti.com/wiki.php/Installation there is:

 Activate TFTP server by adding the following line to
 /etc/inetd.conf and running the "inetconv" command. This will
 create an SMF service, network/tftp/udp6. Don't worry about the
 "udp6" bit-- it listens for both IPv4 and IPv6 connections.

well as it turnes out, you actually SHOULD worry and use 'udp'
instead ... then all will work nicely (on ip4 at least).

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From danmcd at omniti.com  Tue Feb  3 22:58:50 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 17:58:50 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <356582D1FC91784992ABB4265A16ED48886CF466@vEX01.mindstorm-internet.local>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<356582D1FC91784992ABB4265A16ED48886CF466@vEX01.mindstorm-internet.local>
Message-ID: <1A902494-7F29-48B1-B486-38236BEDA78E@omniti.com>

Likely yes.  These drivers were installed manually?  Yeah, that's likely. Have they survived a prior "pkg update"?

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Feb 3, 2015, at 5:38 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dan,
>  
> Immediately after updating, drivers for ntxn ethernet are gone ( they were manually installed, as I had ?funky stuff? with the IGB nics), and need reinstalling.
> Is that normal behavior?
> Rolling back to the before snapshot, and drivers are there again, and traffic is back to normal
>  
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
>  
> Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Dan McDonald
> Verzonden: dinsdag 3 februari 2015 21:36
> Aan: <omnios-discuss at lists.omniti.com>
> Onderwerp: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)
>  
> There is no TL;DR for this.  Please read all of this before you upgrade.
> 
> This is a big one.  I've updated install media AND the repo servers. Users who use "pkg update" will be getting a big wad (I'm not taking chances with missed packages or components).  I'm sorry it took longer than I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed before I released the next bloody update.
> 
> This is r151013-20150203:
> 
> - omnios-build master branch, revision 0c38601
> 
> - NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own section below.
> 
> - OpenSSL is now at 1.0.2.
> 
> - One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we can't catch up until VND is upstreamed from SmartOS).
> 
> - Longstanding bug with GCC 4.4.4's "cc1" not being properly linked. (Thanks Richard Yao of ZFS-on-Linux for catching this.)
> 
> - Curl is now at 7.40.0.
> 
> - NTP dependency fix, from the community. (Thanks to "takashiary".)
> 
> - Git is now at 2.2.1.
> 
> - ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri "lotheac" Tirkkonen.)
> 
> - illumos-omnios master branch, revision aef6850 (last illumos-gate merge 6309835)
> 
> - Several ZFS enhancements, and sometimes followup fixes for them (one, 5531, was what made this release take longer than it should have).
> 
> - dis(1) supports cross-target disassembly  (Illumos 3317)
> 
> - NFS improvements in the authentication cache (Illumos 5509)
> 
> - Several ELF safety improvements from Rich Lowe.
> 
> - kmem_reap() performance improvements
> 
> - preadv() and pwritev()
>  NOTE: KVM's poor handling of these (assuming all the world's Linux) is why we have a cherrypick above.
> 
> - illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in omnios-userland, that will be updated closer to 014's release).
> 
> - Developer prototypes updated to 2015
> 
> - Introducing the linked-ipkg ("lipkg") zone brand.  See below about how this relates to the new pkg(5) changes.
> 
> - Reducing RAM used in managing ZFS cache devices
> 
> - Many new, corrected, and updated man pages.
> 
> - Longstanding mailx(1) overflow fixes now in place.
> 
> 
> NEW FOR 2015 -- updated pkg(5)
> 
> You may have known this from previous bloody releases, but we've been using the same oldish version of pkg(5) since at least r151006 in OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks things like "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old r151006 version of pkg(5).  That changes now.
> 
> If you're interested in source, our new repo for pkg(5) is http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.  It's a downstream of OI Hipster's for now.
> 
> The OpenIndiana Hipster folks have done some work to bring pkg(5) more up to date with its upstream. The new pkg(5) contains many under-the-hood improvements. It also includes two user-visible improvements:
> 
>         - A bit more visible status information during pkg(1) operations.
> 
>         - The ability to link images.
> 
> Linked images is the big one.  Way upstream, linked images are the new default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI Hipster, linked-images are currently disabled.  We decided to take a middle ground:
> 
> 
> THE LINKED IPS BRAND (lipkg)
> 
> Currently deployed ipkg-branded zones stay the same as they have in previous OmniOS releases.  You update them how you're used to updating them:  Either the documented OmniOS way, or "Dan's way", where you mount the newly upgraded BE and then update each zone prior to reboot.
> 
> If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach) you may change its brand the the new linked-ipkg (lipkg) brand.  The default for a new zone is still "ipkg", so if you want lipkg, you must explicitly "set brand=lipkg" in the zonecfg interaction or script.
> 
> How to convert an existing "ipkg" zone to an "lipkg" zone:
> 
> 1.) Install the "lipkg" brand:  "pkg install lipkg".
> 
> 2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone> detach"
> 
> 3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".
> 
> 4.) Attach the zone with the -u flag. pkg(5) needs to run its update check regardless:  "zoneadm -z <zone> attach -u".
> 
> 5.) Boot the zone: "zoneadm -z <zone> boot".
> 
> The zone <zone>'s image will be linked to the global zone's image.
> 
> Once you are running with lipkg zones, new things happen upon subsequent "pkg update"s.  Any packages in the global zone that get updated will also be updated in the lipkg zones at the same time.  This includes more than just the "omnios" publisher as well, which is why lipkg zones aren't for everyone.  If, like me, you tend to currently use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg update" where all zones are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new BE is created, and all of the zones in the new BE are already updated.  Furthermore, if you want to have the semantics of not losing log data during the upgrade, you now merely need to halt the zones, then "pkg update", and reboot.  If your zones are autoboot (which they should be) everything's up and running with shiny new bits.
> 
> Attached as a .txt file is a sample transcript of a linked-image upgrade with a global zone and a non-global zone.
> 
> Please give this one a try, and we'd really appreciate results from:
> 
>         * lipkg-branded zones.
> 
>         * Kayak images.
> 
>         * The ISO installation experience.
> 
>         * ZFS stressing (there may be some residual issues here from upstream, so be warned).
> 
> We will really appreciate the feedback.
> 
> Thanks,
> Dan
> 
>  
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/25e0f08d/attachment.html>

From info at houseofancients.nl  Tue Feb  3 23:05:42 2015
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Tue, 3 Feb 2015 23:05:42 +0000
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <1A902494-7F29-48B1-B486-38236BEDA78E@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<356582D1FC91784992ABB4265A16ED48886CF466@vEX01.mindstorm-internet.local>
	<1A902494-7F29-48B1-B486-38236BEDA78E@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED48886D24F3@vEX01.mindstorm-internet.local>

Hi Dan,

Actually, they have, although those weren?t a major upgrade.
Any chance of these drivers making it within the standard OmniOS Drivers packages ?

These seem to be in Oracle Solaris :

http://docs.oracle.com/cd/E19253-01/816-5177/6mbbc4g8l/index.html

and are HP Nic drivers
genunix: [ID 469746 kern.info] NOTICE: ntxn0 registered
genunix: [ID 805372 kern.info] pcplusmp: pci4040,100 (ntxn) instance 1 irq 0x36 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 2
genunix: [ID 792948 kern.notice] NOTICE: ntxn0: NIC Link is up
genunix: [ID 435574 kern.info] NOTICE: ntxn0 link up, 1000 Mbps, full duplex


Met vriendelijke groet / With kind regards,



               Floris van Essen

Van: Dan McDonald [mailto:danmcd at omniti.com]
Verzonden: dinsdag 3 februari 2015 23:59
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: <omnios-discuss at lists.omniti.com>
Onderwerp: Re: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)

Likely yes.  These drivers were installed manually?  Yeah, that's likely. Have they survived a prior "pkg update"?

Dan

Sent from my iPhone (typos, autocorrect, and all)

On Feb 3, 2015, at 5:38 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl<mailto:info at houseofancients.nl>> wrote:
Hi Dan,

Immediately after updating, drivers for ntxn ethernet are gone ( they were manually installed, as I had ?funky stuff? with the IGB nics), and need reinstalling.
Is that normal behavior?
Rolling back to the before snapshot, and drivers are there again, and traffic is back to normal

Met vriendelijke groet / With kind regards,



               Floris van Essen

Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Dan McDonald
Verzonden: dinsdag 3 februari 2015 21:36
Aan: <omnios-discuss at lists.omniti.com<mailto:omnios-discuss at lists.omniti.com>>
Onderwerp: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)

There is no TL;DR for this.  Please read all of this before you upgrade.

This is a big one.  I've updated install media AND the repo servers. Users who use "pkg update" will be getting a big wad (I'm not taking chances with missed packages or components).  I'm sorry it took longer than I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed before I released the next bloody update.

This is r151013-20150203:

- omnios-build master branch, revision 0c38601

- NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own section below.

- OpenSSL is now at 1.0.2.

- One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we can't catch up until VND is upstreamed from SmartOS).

- Longstanding bug with GCC 4.4.4's "cc1" not being properly linked. (Thanks Richard Yao of ZFS-on-Linux for catching this.)

- Curl is now at 7.40.0.

- NTP dependency fix, from the community. (Thanks to "takashiary".)

- Git is now at 2.2.1.

- ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri "lotheac" Tirkkonen.)

- illumos-omnios master branch, revision aef6850 (last illumos-gate merge 6309835)

- Several ZFS enhancements, and sometimes followup fixes for them (one, 5531, was what made this release take longer than it should have).

- dis(1) supports cross-target disassembly  (Illumos 3317)

- NFS improvements in the authentication cache (Illumos 5509)

- Several ELF safety improvements from Rich Lowe.

- kmem_reap() performance improvements

- preadv() and pwritev()
 NOTE: KVM's poor handling of these (assuming all the world's Linux) is why we have a cherrypick above.

- illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in omnios-userland, that will be updated closer to 014's release).

- Developer prototypes updated to 2015

- Introducing the linked-ipkg ("lipkg") zone brand.  See below about how this relates to the new pkg(5) changes.

- Reducing RAM used in managing ZFS cache devices

- Many new, corrected, and updated man pages.

- Longstanding mailx(1) overflow fixes now in place.


NEW FOR 2015 -- updated pkg(5)

You may have known this from previous bloody releases, but we've been using the same oldish version of pkg(5) since at least r151006 in OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks things like "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old r151006 version of pkg(5).  That changes now.

If you're interested in source, our new repo for pkg(5) is http://github.com/omniti-labs/pkg5 or src.omniti.com<http://src.omniti.com>:~omnios/core/pkg5.  It's a downstream of OI Hipster's for now.

The OpenIndiana Hipster folks have done some work to bring pkg(5) more up to date with its upstream. The new pkg(5) contains many under-the-hood improvements. It also includes two user-visible improvements:

        - A bit more visible status information during pkg(1) operations.

        - The ability to link images.

Linked images is the big one.  Way upstream, linked images are the new default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI Hipster, linked-images are currently disabled.  We decided to take a middle ground:


THE LINKED IPS BRAND (lipkg)

Currently deployed ipkg-branded zones stay the same as they have in previous OmniOS releases.  You update them how you're used to updating them:  Either the documented OmniOS way, or "Dan's way", where you mount the newly upgraded BE and then update each zone prior to reboot.

If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach) you may change its brand the the new linked-ipkg (lipkg) brand.  The default for a new zone is still "ipkg", so if you want lipkg, you must explicitly "set brand=lipkg" in the zonecfg interaction or script.

How to convert an existing "ipkg" zone to an "lipkg" zone:

1.) Install the "lipkg" brand:  "pkg install lipkg".

2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone> detach"

3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".

4.) Attach the zone with the -u flag. pkg(5) needs to run its update check regardless:  "zoneadm -z <zone> attach -u".

5.) Boot the zone: "zoneadm -z <zone> boot".

The zone <zone>'s image will be linked to the global zone's image.

Once you are running with lipkg zones, new things happen upon subsequent "pkg update"s.  Any packages in the global zone that get updated will also be updated in the lipkg zones at the same time.  This includes more than just the "omnios" publisher as well, which is why lipkg zones aren't for everyone.  If, like me, you tend to currently use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg update" where all zones are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new BE is created, and all of the zones in the new BE are already updated.  Furthermore, if you want to have the semantics of not losing log data during the upgrade, you now merely need to halt the zones, then "pkg update", and reboot.  If your zones are autoboot (which they should be) everything's up and running with shiny new bits.

Attached as a .txt file is a sample transcript of a linked-image upgrade with a global zone and a non-global zone.

Please give this one a try, and we'd really appreciate results from:

        * lipkg-branded zones.

        * Kayak images.

        * The ISO installation experience.

        * ZFS stressing (there may be some residual issues here from upstream, so be warned).

We will really appreciate the feedback.

Thanks,
Dan

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl<http://www.houseofancients.nl>

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/59dc3a61/attachment-0001.html>

From danmcd at omniti.com  Tue Feb  3 23:53:35 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 18:53:35 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <356582D1FC91784992ABB4265A16ED48886D24F3@vEX01.mindstorm-internet.local>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<356582D1FC91784992ABB4265A16ED48886CF466@vEX01.mindstorm-internet.local>
	<1A902494-7F29-48B1-B486-38236BEDA78E@omniti.com>
	<356582D1FC91784992ABB4265A16ED48886D24F3@vEX01.mindstorm-internet.local>
Message-ID: <C73CCCEF-4623-49E8-93C7-BE1E41EDCB0C@omniti.com>

If the aren't open source that's CDDL or BSD, they won't.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Feb 3, 2015, at 6:05 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dan,
>  
> Actually, they have, although those weren?t a major upgrade.
> Any chance of these drivers making it within the standard OmniOS Drivers packages ?
>  
> These seem to be in Oracle Solaris :
>  
> http://docs.oracle.com/cd/E19253-01/816-5177/6mbbc4g8l/index.html
>  
> and are HP Nic drivers
> genunix: [ID 469746 kern.info] NOTICE: ntxn0 registered
> genunix: [ID 805372 kern.info] pcplusmp: pci4040,100 (ntxn) instance 1 irq 0x36 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 2
> genunix: [ID 792948 kern.notice] NOTICE: ntxn0: NIC Link is up
> genunix: [ID 435574 kern.info] NOTICE: ntxn0 link up, 1000 Mbps, full duplex
>  
>  
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
>  
> Van: Dan McDonald [mailto:danmcd at omniti.com] 
> Verzonden: dinsdag 3 februari 2015 23:59
> Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
> CC: <omnios-discuss at lists.omniti.com>
> Onderwerp: Re: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)
>  
> Likely yes.  These drivers were installed manually?  Yeah, that's likely. Have they survived a prior "pkg update"?
>  
> Dan
> 
> Sent from my iPhone (typos, autocorrect, and all)
> 
> On Feb 3, 2015, at 5:38 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dan,
>  
> Immediately after updating, drivers for ntxn ethernet are gone ( they were manually installed, as I had ?funky stuff? with the IGB nics), and need reinstalling.
> Is that normal behavior?
> Rolling back to the before snapshot, and drivers are there again, and traffic is back to normal
>  
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
>  
> Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Dan McDonald
> Verzonden: dinsdag 3 februari 2015 21:36
> Aan: <omnios-discuss at lists.omniti.com>
> Onderwerp: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long, please read)
>  
> There is no TL;DR for this.  Please read all of this before you upgrade.
> 
> This is a big one.  I've updated install media AND the repo servers. Users who use "pkg update" will be getting a big wad (I'm not taking chances with missed packages or components).  I'm sorry it took longer than I expected, but there was an illumos-gate ZFS bug (5531) I wanted fixed before I released the next bloody update.
> 
> This is r151013-20150203:
> 
> - omnios-build master branch, revision 0c38601
> 
> - NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own section below.
> 
> - OpenSSL is now at 1.0.2.
> 
> - One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because we can't catch up until VND is upstreamed from SmartOS).
> 
> - Longstanding bug with GCC 4.4.4's "cc1" not being properly linked. (Thanks Richard Yao of ZFS-on-Linux for catching this.)
> 
> - Curl is now at 7.40.0.
> 
> - NTP dependency fix, from the community. (Thanks to "takashiary".)
> 
> - Git is now at 2.2.1.
> 
> - ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to Lauri "lotheac" Tirkkonen.)
> 
> - illumos-omnios master branch, revision aef6850 (last illumos-gate merge 6309835)
> 
> - Several ZFS enhancements, and sometimes followup fixes for them (one, 5531, was what made this release take longer than it should have).
> 
> - dis(1) supports cross-target disassembly  (Illumos 3317)
> 
> - NFS improvements in the authentication cache (Illumos 5509)
> 
> - Several ELF safety improvements from Rich Lowe.
> 
> - kmem_reap() performance improvements
> 
> - preadv() and pwritev()
>  NOTE: KVM's poor handling of these (assuming all the world's Linux) is why we have a cherrypick above.
> 
> - illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy in omnios-userland, that will be updated closer to 014's release).
> 
> - Developer prototypes updated to 2015
> 
> - Introducing the linked-ipkg ("lipkg") zone brand.  See below about how this relates to the new pkg(5) changes.
> 
> - Reducing RAM used in managing ZFS cache devices
> 
> - Many new, corrected, and updated man pages.
> 
> - Longstanding mailx(1) overflow fixes now in place.
> 
> 
> NEW FOR 2015 -- updated pkg(5)
> 
> You may have known this from previous bloody releases, but we've been using the same oldish version of pkg(5) since at least r151006 in OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks things like "zoneadm -z <zone> attach -u".  When a release came, we reverted to the old r151006 version of pkg(5).  That changes now.
> 
> If you're interested in source, our new repo for pkg(5) is http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.  It's a downstream of OI Hipster's for now.
> 
> The OpenIndiana Hipster folks have done some work to bring pkg(5) more up to date with its upstream. The new pkg(5) contains many under-the-hood improvements. It also includes two user-visible improvements:
> 
>         - A bit more visible status information during pkg(1) operations.
> 
>         - The ability to link images.
> 
> Linked images is the big one.  Way upstream, linked images are the new default for ipkg-branded zones (i.e. the zones all OmniOS folks use). In OI Hipster, linked-images are currently disabled.  We decided to take a middle ground:
> 
> 
> THE LINKED IPS BRAND (lipkg)
> 
> Currently deployed ipkg-branded zones stay the same as they have in previous OmniOS releases.  You update them how you're used to updating them:  Either the documented OmniOS way, or "Dan's way", where you mount the newly upgraded BE and then update each zone prior to reboot.
> 
> If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone> detach) you may change its brand the the new linked-ipkg (lipkg) brand.  The default for a new zone is still "ipkg", so if you want lipkg, you must explicitly "set brand=lipkg" in the zonecfg interaction or script.
> 
> How to convert an existing "ipkg" zone to an "lipkg" zone:
> 
> 1.) Install the "lipkg" brand:  "pkg install lipkg".
> 
> 2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z <zone> detach"
> 
> 3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".
> 
> 4.) Attach the zone with the -u flag. pkg(5) needs to run its update check regardless:  "zoneadm -z <zone> attach -u".
> 
> 5.) Boot the zone: "zoneadm -z <zone> boot".
> 
> The zone <zone>'s image will be linked to the global zone's image.
> 
> Once you are running with lipkg zones, new things happen upon subsequent "pkg update"s.  Any packages in the global zone that get updated will also be updated in the lipkg zones at the same time.  This includes more than just the "omnios" publisher as well, which is why lipkg zones aren't for everyone.  If, like me, you tend to currently use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg update" where all zones are lipkg is the equivalent of an automatic "Dan's way" upgrade.  The new BE is created, and all of the zones in the new BE are already updated.  Furthermore, if you want to have the semantics of not losing log data during the upgrade, you now merely need to halt the zones, then "pkg update", and reboot.  If your zones are autoboot (which they should be) everything's up and running with shiny new bits.
> 
> Attached as a .txt file is a sample transcript of a linked-image upgrade with a global zone and a non-global zone.
> 
> Please give this one a try, and we'd really appreciate results from:
> 
>         * lipkg-branded zones.
> 
>         * Kayak images.
> 
>         * The ISO installation experience.
> 
>         * ZFS stressing (there may be some residual issues here from upstream, so be warned).
> 
> We will really appreciate the feedback.
> 
> Thanks,
> Dan
> 
>  
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
> 
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150203/b3bc2ae9/attachment-0001.html>

From danmcd at omniti.com  Wed Feb  4 01:02:40 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 3 Feb 2015 20:02:40 -0500
Subject: [OmniOS-discuss] in.tftpd not working and how to fix it
In-Reply-To: <alpine.DEB.2.10.1502032339230.14595@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1502032339230.14595@froburg.oetiker.ch>
Message-ID: <2470DC98-C8FF-423F-905A-2ECEAAAD021A@omniti.com>

I'll make sure the wiki gets fixed this week.

Thanks,
Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Feb 3, 2015, at 5:46 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> 
> caveat: due to the kvm performance problem I am still running 010
> ... but maybe this is also a problem on more recent versions ...
> 
> so here goes:
> 
> after upgrading to 010 the tftp service on our omnios box stopped
> working properly ... it served a few bytes of the request and then
> timed out ...
> 
> we investigated for a bit and then moved our tftp service to linux.
> 
> Today I ran into the problem again and found the solution:
> 
> on http://omnios.omniti.com/wiki.php/Installation there is:
> 
> Activate TFTP server by adding the following line to
> /etc/inetd.conf and running the "inetconv" command. This will
> create an SMF service, network/tftp/udp6. Don't worry about the
> "udp6" bit-- it listens for both IPv4 and IPv6 connections.
> 
> well as it turnes out, you actually SHOULD worry and use 'udp'
> instead ... then all will work nicely (on ip4 at least).
> 
> cheers
> tobi
> 
> -- 
> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From chip at innovates.com  Wed Feb  4 14:15:00 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 4 Feb 2015 08:15:00 -0600
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <B15C3F78-7148-4FFE-B14F-70A2B5CC0DB1@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
	<D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
	<CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>
	<B15C3F78-7148-4FFE-B14F-70A2B5CC0DB1@omniti.com>
Message-ID: <CALeZrrQOspFXC4kykHjHVxTMKDD2MONkBxPnMN8Y5F32NwbgpQ@mail.gmail.com>

Maybe I haven't had enough coffee this morning or I'm missing something.

Where should I set the pkg publisher to point to?

I tried:
http://pkg.omniti.com/omnios/r151013/
and
http://pkg.omniti.com/omnios/r151013-20150203/

Neither works.

-Chip

On Tue, Feb 3, 2015 at 3:08 PM, Dan McDonald <danmcd at omniti.com> wrote:

>
> > On Feb 3, 2015, at 4:07 PM, Schweiss, Chip <chip at innovates.com> wrote:
> >
> > Good to know.  Regardless, I will run bloody until I'm ready to go live
> on this system.   Hopefully I can contribute some valuable information,
> even if it is in the form of crash dumps.
>
> Crash dumps are appreciated.  Also, you saw what else (packaging) I'm
> interested in for this particular one.
>
> Thanks!
> Dan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150204/12e8e34e/attachment.html>

From jesus at omniti.com  Wed Feb  4 15:00:35 2015
From: jesus at omniti.com (Theo Schlossnagle)
Date: Wed, 4 Feb 2015 10:00:35 -0500
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <CALeZrrQOspFXC4kykHjHVxTMKDD2MONkBxPnMN8Y5F32NwbgpQ@mail.gmail.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
	<D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
	<CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>
	<B15C3F78-7148-4FFE-B14F-70A2B5CC0DB1@omniti.com>
	<CALeZrrQOspFXC4kykHjHVxTMKDD2MONkBxPnMN8Y5F32NwbgpQ@mail.gmail.com>
Message-ID: <CACLsApv2h0b5E+agAbW_Owb0-M10-v1uuXCsKs6WrxW-esOANg@mail.gmail.com>

http://pkg.omniti.com/omnios/bloody/

On Wed, Feb 4, 2015 at 9:15 AM, Schweiss, Chip <chip at innovates.com> wrote:

> Maybe I haven't had enough coffee this morning or I'm missing something.
>
> Where should I set the pkg publisher to point to?
>
> I tried:
> http://pkg.omniti.com/omnios/r151013/
> and
> http://pkg.omniti.com/omnios/r151013-20150203/
>
> Neither works.
>
> -Chip
>
> On Tue, Feb 3, 2015 at 3:08 PM, Dan McDonald <danmcd at omniti.com> wrote:
>
>>
>> > On Feb 3, 2015, at 4:07 PM, Schweiss, Chip <chip at innovates.com> wrote:
>> >
>> > Good to know.  Regardless, I will run bloody until I'm ready to go live
>> on this system.   Hopefully I can contribute some valuable information,
>> even if it is in the form of crash dumps.
>>
>> Crash dumps are appreciated.  Also, you saw what else (packaging) I'm
>> interested in for this particular one.
>>
>> Thanks!
>> Dan
>>
>>
>


-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150204/56456fd8/attachment.html>

From chip at innovates.com  Wed Feb  4 16:12:33 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 4 Feb 2015 10:12:33 -0600
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
 please read)
In-Reply-To: <CACLsApv2h0b5E+agAbW_Owb0-M10-v1uuXCsKs6WrxW-esOANg@mail.gmail.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
	<CACLsApttcLsqYRkT0NfnkDSV4AG=9UzEdin5JRu_JFyLHu54wQ@mail.gmail.com>
	<CALeZrrTpgPN8qZxoUxfcpWz-L9xygc9sNmRa4F1bJXwvFiy_Mw@mail.gmail.com>
	<D52BDAF7-1FDD-4242-A6FC-ED10DCCF882F@omniti.com>
	<CALeZrrSoVzz2NTiXoAXuYs2Mgw9BEpY3jvM0E6+Dr9iX+rzG_Q@mail.gmail.com>
	<B15C3F78-7148-4FFE-B14F-70A2B5CC0DB1@omniti.com>
	<CALeZrrQOspFXC4kykHjHVxTMKDD2MONkBxPnMN8Y5F32NwbgpQ@mail.gmail.com>
	<CACLsApv2h0b5E+agAbW_Owb0-M10-v1uuXCsKs6WrxW-esOANg@mail.gmail.com>
Message-ID: <CALeZrrRuj3QfDazoOoRMC+eCinVdFcpdsLVnhu=30M4eBUvUeA@mail.gmail.com>

On Wed, Feb 4, 2015 at 9:00 AM, Theo Schlossnagle <jesus at omniti.com> wrote:

> http://pkg.omniti.com/omnios/bloody/
>

That did it.  Thanks!

-Chip

>
> On Wed, Feb 4, 2015 at 9:15 AM, Schweiss, Chip <chip at innovates.com> wrote:
>
>> Maybe I haven't had enough coffee this morning or I'm missing something.
>>
>> Where should I set the pkg publisher to point to?
>>
>> I tried:
>> http://pkg.omniti.com/omnios/r151013/
>> and
>> http://pkg.omniti.com/omnios/r151013-20150203/
>>
>> Neither works.
>>
>> -Chip
>>
>> On Tue, Feb 3, 2015 at 3:08 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>
>>>
>>> > On Feb 3, 2015, at 4:07 PM, Schweiss, Chip <chip at innovates.com> wrote:
>>> >
>>> > Good to know.  Regardless, I will run bloody until I'm ready to go
>>> live on this system.   Hopefully I can contribute some valuable
>>> information, even if it is in the form of crash dumps.
>>>
>>> Crash dumps are appreciated.  Also, you saw what else (packaging) I'm
>>> interested in for this particular one.
>>>
>>> Thanks!
>>> Dan
>>>
>>>
>>
>
>
> --
>
> Theo Schlossnagle
>
> http://omniti.com/is/theo-schlossnagle
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150204/1d1e4848/attachment.html>

From jimklimov at cos.ru  Fri Feb  6 13:25:38 2015
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 06 Feb 2015 14:25:38 +0100
Subject: [OmniOS-discuss] FINALLY - OmniOS bloody is now updated. (long,
	please read)
In-Reply-To: <E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
References: <09C29A92-D7F9-48A9-90DE-4C535DE220ED@omniti.com>
	<E9C1B608-DC2E-499F-AA88-B2845D85BA2A@omniti.com>
Message-ID: <481ACD53-C54C-4CBD-8EA8-D30F706026FA@cos.ru>

3 ??????? 2015??. 21:35:30 CET, Dan McDonald <danmcd at omniti.com> ?????:
>There is no TL;DR for this.  Please read all of this before you
>upgrade.
>
>This is a big one.  I've updated install media AND the repo servers.
>Users who use "pkg update" will be getting a big wad (I'm not taking
>chances with missed packages or components).  I'm sorry it took longer
>than I expected, but there was an illumos-gate ZFS bug (5531) I wanted
>fixed before I released the next bloody update.
>
>This is r151013-20150203:
>
>- omnios-build master branch, revision 0c38601
>
>- NEW UPDATE to pkg(5).  This carries a LOT with it, and gets its own
>section below.
>
>- OpenSSL is now at 1.0.2.
>
>- One bugfix to illumos-kvm-cmd that needed to be cherrypicked (because
>we can't catch up until VND is upstreamed from SmartOS).
>
>- Longstanding bug with GCC 4.4.4's "cc1" not being properly linked.
>(Thanks Richard Yao of ZFS-on-Linux for catching this.)
>
>- Curl is now at 7.40.0.
>
>- NTP dependency fix, from the community. (Thanks to "takashiary".)
>
>- Git is now at 2.2.1.
>
>- ncurses have their auxiliary libs now in /usr/gnu/lib.  (Thanks to
>Lauri "lotheac" Tirkkonen.)
>
>- illumos-omnios master branch, revision aef6850 (last illumos-gate
>merge 6309835)
>
>- Several ZFS enhancements, and sometimes followup fixes for them (one,
>5531, was what made this release take longer than it should have).
>
>- dis(1) supports cross-target disassembly  (Illumos 3317)
>
>- NFS improvements in the authentication cache (Illumos 5509)
>
>- Several ELF safety improvements from Rich Lowe.
>
>- kmem_reap() performance improvements
>
>- preadv() and pwritev()
>NOTE: KVM's poor handling of these (assuming all the world's Linux) is
>why we have a cherrypick above.
>
>- illumos-side hwdata (PCI & USB) updates.  (NOTE: OmniOS keeps a copy
>in omnios-userland, that will be updated closer to 014's release).
>
>- Developer prototypes updated to 2015
>
>- Introducing the linked-ipkg ("lipkg") zone brand.  See below about
>how this relates to the new pkg(5) changes.
>
>- Reducing RAM used in managing ZFS cache devices
>
>- Many new, corrected, and updated man pages.
>
>- Longstanding mailx(1) overflow fixes now in place.
>
>
>NEW FOR 2015 -- updated pkg(5)
>
>You may have known this from previous bloody releases, but we've been
>using the same oldish version of pkg(5) since at least r151006 in
>OmniOS.  Bloody releases have had a half-finished pkg(5) that breaks
>things like "zoneadm -z <zone> attach -u".  When a release came, we
>reverted to the old r151006 version of pkg(5).  That changes now.
>
>If you're interested in source, our new repo for pkg(5) is
>http://github.com/omniti-labs/pkg5 or src.omniti.com:~omnios/core/pkg5.
> It's a downstream of OI Hipster's for now.
>
>The OpenIndiana Hipster folks have done some work to bring pkg(5) more
>up to date with its upstream. The new pkg(5) contains many
>under-the-hood improvements. It also includes two user-visible
>improvements:
>
>	- A bit more visible status information during pkg(1) operations.
>
>	- The ability to link images.
>
>Linked images is the big one.  Way upstream, linked images are the new
>default for ipkg-branded zones (i.e. the zones all OmniOS folks use).
>In OI Hipster, linked-images are currently disabled.  We decided to
>take a middle ground:
>
>
>THE LINKED IPS BRAND (lipkg)
>
>Currently deployed ipkg-branded zones stay the same as they have in
>previous OmniOS releases.  You update them how you're used to updating
>them:  Either the documented OmniOS way, or "Dan's way", where you
>mount the newly upgraded BE and then update each zone prior to reboot.
>
>If you detach a zone:  (zoneadm -z <zone> halt ; zoneadm -z <zone>
>detach) you may change its brand the the new linked-ipkg (lipkg) brand.
>The default for a new zone is still "ipkg", so if you want lipkg, you
>must explicitly "set brand=lipkg" in the zonecfg interaction or script.
>
>How to convert an existing "ipkg" zone to an "lipkg" zone:
>
>1.) Install the "lipkg" brand:  "pkg install lipkg".
>
>2.) Halt and detach the zone.  "zoneadm -z <zone> halt ; zoneadm -z
><zone> detach"
>
>3.) Change the zone's brand:  "zonecfg -z <zone> set brand=lipkg".
>
>4.) Attach the zone with the -u flag. pkg(5) needs to run its update
>check regardless:  "zoneadm -z <zone> attach -u".
>
>5.) Boot the zone: "zoneadm -z <zone> boot".
>
>The zone <zone>'s image will be linked to the global zone's image.
>
>Once you are running with lipkg zones, new things happen upon
>subsequent "pkg update"s.  Any packages in the global zone that get
>updated will also be updated in the lipkg zones at the same time.  This
>includes more than just the "omnios" publisher as well, which is why
>lipkg zones aren't for everyone.  If, like me, you tend to currently
>use the "Dan's way" of upgrading, lipkg zones will save you time.  "pkg
>update" where all zones are lipkg is the equivalent of an automatic
>"Dan's way" upgrade.  The new BE is created, and all of the zones in
>the new BE are already updated.  Furthermore, if you want to have the
>semantics of not losing log data during the upgrade, you now merely
>need to halt the zones, then "pkg update", and reboot.  If your zones
>are autoboot (which they should be) everything's up and running with
>shiny new bits.
>
>Attached as a .txt file is a sample transcript of a linked-image
>upgrade with a global zone and a non-global zone.
>
>Please give this one a try, and we'd really appreciate results from:
>
>	* lipkg-branded zones.
>
>	* Kayak images.
>
>	* The ISO installation experience.
>
>	* ZFS stressing (there may be some residual issues here from upstream,
>so be warned).
>
>We will really appreciate the feedback.
>
>Thanks,
>Dan
>
>
>
>------------------------------------------------------------------------
>
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Upgrade of my split-root omnios installation with my scripts that manage these more complicated BE's went almost well, the (shared) /var/log dataset needed to be (lofs-)mounted into the cloned BE for pkg to succeed.

Relevant updates to beadm-update.sh should now be available in my github at https://github.com/jimklimov/illumos-splitroot-scripts if anyone else does those setups ;)

If anyone gets around to port this logic into proper filesystem smf, zfs clone and beadm toolset - i'd be happy ;)

Thanks,
Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From contact at jacobvosmaer.nl  Sat Feb  7 23:16:48 2015
From: contact at jacobvosmaer.nl (Jacob Vosmaer)
Date: Sun, 8 Feb 2015 00:16:48 +0100
Subject: [OmniOS-discuss] Corrupted ZFS metadata?
In-Reply-To: <AF33E9BC-99D5-4652-8FE2-0FA973341251@marzocchi.net>
References: <5C57DEC9-2FE9-4047-BB09-8B489A425BC1@marzocchi.net>
	<95AEF4F6-CFFC-4B24-8AB1-70B32DBDF616@omniti.com>
	<7E566302-A0D7-4D27-81EB-A22C2A17C00B@marzocchi.net>
	<4DE2883C-F7DC-4610-A3ED-470786E5C58F@omniti.com>
	<AF33E9BC-99D5-4652-8FE2-0FA973341251@marzocchi.net>
Message-ID: <CAMcGgP34Tcn64JsS-Tc0JYSnZvXAVDR8xbdSwvJDQCQ6=JjP3g@mail.gmail.com>

Hi,

I use OS X 10.10.x clients to access ZFS filesystems on OmniOS r151012 via
NFS. Not quite the same as Olaf but I do sometimes get weird 'corrupted
file' errors from OS X.

Olaf, have you tried using 'cp' to copy one of the seemingly corrupted
files from your OmniOS box to a local OS X volume before opening the file
in OS X? Another fun one for me is looking for extended attributes with 'ls
-l@' and clearing them with 'xattr -c'. Not sure if it ever fixes anything
but it's something one can poke around with.

2015-01-29 22:32 GMT+01:00 Olaf Marzocchi <lists at marzocchi.net>:

>
> > Il giorno 29/gen/2015, alle ore 22:29, Dan McDonald <danmcd at omniti.com>
> ha scritto:
> >
> >
> >> On Jan 29, 2015, at 4:27 PM, Olaf Marzocchi <lists at marzocchi.net>
> wrote:
> >>
> >> Scrubs are performed biweekly by a cron job that should send me the
> output in case of errors. I never got anything and indeed I still see no
> errors in zpool status.
> >>
> >> In case it happens again I won?t overwrite the files and I?ll contact
> the list to investigate further, I suppose it?s late now to do anything.
> >
> > One last question:  which OmniOS release are you running?
>
> Latest one, non-bloody: r151012
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150208/246a261f/attachment.html>

From lists at marzocchi.net  Sun Feb  8 15:15:46 2015
From: lists at marzocchi.net (Olaf Marzocchi)
Date: Sun, 8 Feb 2015 16:15:46 +0100
Subject: [OmniOS-discuss] Corrupted ZFS metadata?
In-Reply-To: <CAMcGgP34Tcn64JsS-Tc0JYSnZvXAVDR8xbdSwvJDQCQ6=JjP3g@mail.gmail.com>
References: <5C57DEC9-2FE9-4047-BB09-8B489A425BC1@marzocchi.net>
	<95AEF4F6-CFFC-4B24-8AB1-70B32DBDF616@omniti.com>
	<7E566302-A0D7-4D27-81EB-A22C2A17C00B@marzocchi.net>
	<4DE2883C-F7DC-4610-A3ED-470786E5C58F@omniti.com>
	<AF33E9BC-99D5-4652-8FE2-0FA973341251@marzocchi.net>
	<CAMcGgP34Tcn64JsS-Tc0JYSnZvXAVDR8xbdSwvJDQCQ6=JjP3g@mail.gmail.com>
Message-ID: <70278C2F-490F-42DD-908F-E103BABA7519@marzocchi.net>

Hi,
I?ll keep it in mind, but also next time I will try accessing the files via SMB, to verify if the issue is somehow related to netatalk.
If you have an idea about how to find those ?falsely corrupted? files let me know, so that I don?t have to wait until I find one by chance.

Thanks
Olaf



> Il giorno 08/feb/2015, alle ore 00:16, Jacob Vosmaer <contact at jacobvosmaer.nl> ha scritto:
> 
> Hi,
> 
> I use OS X 10.10.x clients to access ZFS filesystems on OmniOS r151012 via NFS. Not quite the same as Olaf but I do sometimes get weird 'corrupted file' errors from OS X.
> 
> Olaf, have you tried using 'cp' to copy one of the seemingly corrupted files from your OmniOS box to a local OS X volume before opening the file in OS X? Another fun one for me is looking for extended attributes with 'ls -l@' and clearing them with 'xattr -c'. Not sure if it ever fixes anything but it's something one can poke around with.
> 
> 2015-01-29 22:32 GMT+01:00 Olaf Marzocchi <lists at marzocchi.net <mailto:lists at marzocchi.net>>:
> 
> > Il giorno 29/gen/2015, alle ore 22:29, Dan McDonald <danmcd at omniti.com <mailto:danmcd at omniti.com>> ha scritto:
> >
> >
> >> On Jan 29, 2015, at 4:27 PM, Olaf Marzocchi <lists at marzocchi.net <mailto:lists at marzocchi.net>> wrote:
> >>
> >> Scrubs are performed biweekly by a cron job that should send me the output in case of errors. I never got anything and indeed I still see no errors in zpool status.
> >>
> >> In case it happens again I won?t overwrite the files and I?ll contact the list to investigate further, I suppose it?s late now to do anything.
> >
> > One last question:  which OmniOS release are you running?
> 
> Latest one, non-bloody: r151012
> 
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150208/5c68d76b/attachment.html>

From asc1111 at gmail.com  Mon Feb  9 17:27:41 2015
From: asc1111 at gmail.com (Aaron Curry)
Date: Mon, 9 Feb 2015 10:27:41 -0700
Subject: [OmniOS-discuss] File Server Crash
Message-ID: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>

Hi all,

We recently set up an OmniOS file server to serve our SMB clients. The
server paniced over the weekend. I was hoping someone here would help us
interpret the panic info and possibly point us towards a resolution. The
stack dump is:

                panicstr = BAD TRAP: type=e (#pf Page fault)
rp=ffffff007c6e78a0 addr=18 occurred in module "nsmb" due to a NULL pointer
dereference
                panicstack = unix:die+df () | unix:trap+db3 () |
unix:cmntrap+e6 () | nsmb:nbssn_peekhdr+75 () | nsmb:nbssn_recv+7f () |
nsmb:smb_nbst_recv+a1 () | nsmb:smb_iod_recv1+46 () |
nsmb:smb_iod_recvall+65 () | nsmb:smb_iod_vc_work+c6 () |
nsmb:smb_usr_iod_work+15c () | nsmb:nsmb_ioctl+d9 () |
genunix:cdev_ioctl+39 () | specfs:spec_ioctl+60 () | genunix:fop_ioctl+55
() | genunix:ioctl+9b () | unix:brand_sys_sysenter+1c9 () |


I can provide the full crash dump if it will help... once it comes back up.
Looks like it crashed again while I was writing this.

Thanks,

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150209/c3350567/attachment.html>

From Robert.Brock at 2hoffshore.com  Mon Feb  9 17:41:35 2015
From: Robert.Brock at 2hoffshore.com (Robert A. Brock)
Date: Mon, 9 Feb 2015 17:41:35 +0000
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
Message-ID: <2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>

>From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Aaron Curry
>Sent: 09 February 2015 17:28
>To: omnios-discuss at lists.omniti.com
>Subject: [OmniOS-discuss] File Server Crash
>
>Hi all,
>
>We recently set up an OmniOS file server to serve our SMB clients. The server paniced over the weekend. I was hoping someone here would help us interpret the panic info and possibly point us towards a resolution. The stack dump is:
>
> ? ? ? ? ? ? ? panicstr = BAD TRAP: type=e (#pf Page fault) rp=ffffff007c6e78a0 addr=18 occurred in module "nsmb" due to a NULL pointer dereference
>? ? ? ? ? ? ? ? panicstack = unix:die+df () | unix:trap+db3 () | unix:cmntrap+e6 () | nsmb:nbssn_peekhdr+75 () | nsmb:nbssn_recv+7f () | nsmb:smb_nbst_recv+a1 () | nsmb:smb_iod_recv1+46 () | nsmb:smb_iod_recvall+65 () | nsmb:smb_iod_vc_work+c6 () | nsmb:smb_usr_iod_work+15c () | nsmb:nsmb_ioctl+d9 () | >genunix:cdev_ioctl+39 () | specfs:spec_ioctl+60 () | genunix:fop_ioctl+55 () | genunix:ioctl+9b () | unix:brand_sys_sysenter+1c9 () |
>
>
>I can provide the full crash dump if it will help... once it comes back up. Looks like it crashed again while I was writing this.
>
>Thanks,
>
>Aaron

Smells like this: https://github.com/Nexenta/illumos-nexenta/commit/9a60a9d168313d9c350f813abccc6320879ad9b5

Not sure if this fix has made it to OmniOS.

From asc1111 at gmail.com  Mon Feb  9 18:12:23 2015
From: asc1111 at gmail.com (Aaron Curry)
Date: Mon, 9 Feb 2015 11:12:23 -0700
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
Message-ID: <CAOqBcP83CHMw-QXpkwv230SrpN48ZXFhGLAjET+A+2LnNdTEBA@mail.gmail.com>

Robert, thanks for looking at this.

Looks like that fix was committed to OmniOS last September:

https://github.com/omniti-labs/illumos-omnios/commit/df1c321ce1e8da6abd16cf562f5cef4ecd9e89c6

Does that mean it made it into r151012 which was released in October? This
server is currently running r151010. Can we assume that an update to
r151012 will fix the issue?

Thanks again,

Aaron

On Mon, Feb 9, 2015 at 10:41 AM, Robert A. Brock <
Robert.Brock at 2hoffshore.com> wrote:

> >From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On
> Behalf Of Aaron Curry
> >Sent: 09 February 2015 17:28
> >To: omnios-discuss at lists.omniti.com
> >Subject: [OmniOS-discuss] File Server Crash
> >
> >Hi all,
> >
> >We recently set up an OmniOS file server to serve our SMB clients. The
> server paniced over the weekend. I was hoping someone here would help us
> interpret the panic info and possibly point us towards a resolution. The
> stack dump is:
> >
> >               panicstr = BAD TRAP: type=e (#pf Page fault)
> rp=ffffff007c6e78a0 addr=18 occurred in module "nsmb" due to a NULL pointer
> dereference
> >                panicstack = unix:die+df () | unix:trap+db3 () |
> unix:cmntrap+e6 () | nsmb:nbssn_peekhdr+75 () | nsmb:nbssn_recv+7f () |
> nsmb:smb_nbst_recv+a1 () | nsmb:smb_iod_recv1+46 () |
> nsmb:smb_iod_recvall+65 () | nsmb:smb_iod_vc_work+c6 () |
> nsmb:smb_usr_iod_work+15c () | nsmb:nsmb_ioctl+d9 () |
> >genunix:cdev_ioctl+39 () | specfs:spec_ioctl+60 () | genunix:fop_ioctl+55
> () | genunix:ioctl+9b () | unix:brand_sys_sysenter+1c9 () |
> >
> >
> >I can provide the full crash dump if it will help... once it comes back
> up. Looks like it crashed again while I was writing this.
> >
> >Thanks,
> >
> >Aaron
>
> Smells like this:
> https://github.com/Nexenta/illumos-nexenta/commit/9a60a9d168313d9c350f813abccc6320879ad9b5
>
> Not sure if this fix has made it to OmniOS.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150209/20f0d35c/attachment-0001.html>

From mir at miras.org  Mon Feb  9 18:41:49 2015
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 9 Feb 2015 19:41:49 +0100
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <CAOqBcP83CHMw-QXpkwv230SrpN48ZXFhGLAjET+A+2LnNdTEBA@mail.gmail.com>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
	<CAOqBcP83CHMw-QXpkwv230SrpN48ZXFhGLAjET+A+2LnNdTEBA@mail.gmail.com>
Message-ID: <20150209194149.76f210d0@sleipner.datanom.net>

On Mon, 9 Feb 2015 11:12:23 -0700
Aaron Curry <asc1111 at gmail.com> wrote:

> 
> Does that mean it made it into r151012 which was released in October? This
> server is currently running r151010. Can we assume that an update to
> r151012 will fix the issue?
> 
Yup. https://github.com/omniti-labs/illumos-omnios/commits/r151012
Scroll down to 'Commits on Sep 8, 2014'

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
They also surf who only stand on waves.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150209/647d8fc5/attachment.bin>

From danmcd at omniti.com  Mon Feb  9 19:07:33 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 9 Feb 2015 14:07:33 -0500
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
Message-ID: <AEE0AF04-977A-4C27-8686-64C5A47255D9@omniti.com>


> On Feb 9, 2015, at 12:41 PM, Robert A. Brock <Robert.Brock at 2hoffshore.com> wrote:
> 
> 
> Smells like this: https://github.com/Nexenta/illumos-nexenta/commit/9a60a9d168313d9c350f813abccc6320879ad9b5
> 
> Not sure if this fix has made it to OmniOS.

That fix was specially backported to r151012 just before that branch closed for its release.  If the original poster is running long-term Support or even r151010, that would explain it.

If this is happening on r151012 or bloody, please provide the crash dump, Aaron, as it's NOT the aforementioned bug.

Dan


From asc1111 at gmail.com  Mon Feb  9 19:41:01 2015
From: asc1111 at gmail.com (Aaron Curry)
Date: Mon, 9 Feb 2015 12:41:01 -0700
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <AEE0AF04-977A-4C27-8686-64C5A47255D9@omniti.com>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
	<AEE0AF04-977A-4C27-8686-64C5A47255D9@omniti.com>
Message-ID: <CAOqBcP8PsCX4_SAFDmu-g51Ti6FdN3N9E2Z2uswMddxG-+ppDQ@mail.gmail.com>

Thanks for confirming that. We will be updating tonight. Hopefully we don't
see this again!

Aaron

On Mon, Feb 9, 2015 at 12:07 PM, Dan McDonald <danmcd at omniti.com> wrote:

>
> > On Feb 9, 2015, at 12:41 PM, Robert A. Brock <
> Robert.Brock at 2hoffshore.com> wrote:
> >
> >
> > Smells like this:
> https://github.com/Nexenta/illumos-nexenta/commit/9a60a9d168313d9c350f813abccc6320879ad9b5
> >
> > Not sure if this fix has made it to OmniOS.
>
> That fix was specially backported to r151012 just before that branch
> closed for its release.  If the original poster is running long-term
> Support or even r151010, that would explain it.
>
> If this is happening on r151012 or bloody, please provide the crash dump,
> Aaron, as it's NOT the aforementioned bug.
>
> Dan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150209/0070f512/attachment.html>

From danmcd at omniti.com  Mon Feb  9 19:41:35 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 9 Feb 2015 14:41:35 -0500
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <CAOqBcP8PsCX4_SAFDmu-g51Ti6FdN3N9E2Z2uswMddxG-+ppDQ@mail.gmail.com>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
	<AEE0AF04-977A-4C27-8686-64C5A47255D9@omniti.com>
	<CAOqBcP8PsCX4_SAFDmu-g51Ti6FdN3N9E2Z2uswMddxG-+ppDQ@mail.gmail.com>
Message-ID: <8957ABD5-4EC2-4D1F-A1C1-C4445A06C3C3@omniti.com>


> On Feb 9, 2015, at 2:41 PM, Aaron Curry <asc1111 at gmail.com> wrote:
> 
> Thanks for confirming that. We will be updating tonight. Hopefully we don't see this again!

Can you tell me what you *were* running?  (For a reference point?)

Thanks,
Dan


From asc1111 at gmail.com  Mon Feb  9 20:27:20 2015
From: asc1111 at gmail.com (Aaron Curry)
Date: Mon, 9 Feb 2015 13:27:20 -0700
Subject: [OmniOS-discuss] File Server Crash
In-Reply-To: <8957ABD5-4EC2-4D1F-A1C1-C4445A06C3C3@omniti.com>
References: <CAOqBcP9jQjpeG_9Q=TwbdJ+_DVU2aE0YeffbnxHHy6Ao8Oc=Hw@mail.gmail.com>
	<2859482C466CCA42AD9B84B9F56212300232BC2A@2H199.2hukwok2.local>
	<AEE0AF04-977A-4C27-8686-64C5A47255D9@omniti.com>
	<CAOqBcP8PsCX4_SAFDmu-g51Ti6FdN3N9E2Z2uswMddxG-+ppDQ@mail.gmail.com>
	<8957ABD5-4EC2-4D1F-A1C1-C4445A06C3C3@omniti.com>
Message-ID: <CAOqBcP8fxVgMFmoO05e4gJb44bWxTruAs9g66TW-2aL8sDzbrw@mail.gmail.com>

This server is currently running r151010.

Aaron

On Mon, Feb 9, 2015 at 12:41 PM, Dan McDonald <danmcd at omniti.com> wrote:

>
> > On Feb 9, 2015, at 2:41 PM, Aaron Curry <asc1111 at gmail.com> wrote:
> >
> > Thanks for confirming that. We will be updating tonight. Hopefully we
> don't see this again!
>
> Can you tell me what you *were* running?  (For a reference point?)
>
> Thanks,
> Dan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150209/89727563/attachment.html>

From danmcd at omniti.com  Tue Feb 10 02:22:33 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 9 Feb 2015 21:22:33 -0500
Subject: [OmniOS-discuss] Small update to UNZIP covers CVE 2014-9636
Message-ID: <87BB14EB-DFF5-4FDF-A184-7C7FE6468F18@omniti.com>

Thanks to Marissa Murphy for patching this.  It's now available for "pkg update" on all supported repos:  r151006/LTS, r151010/last-Stable, r151012/Stable, and r151013/bloody.

Won't require a reboot, nor service restarts since these are just command replacements.

Dan


From marc.edwards at siliconcloudinternational.com  Tue Feb 10 23:20:34 2015
From: marc.edwards at siliconcloudinternational.com (J Marc Edwards)
Date: Tue, 10 Feb 2015 18:20:34 -0500
Subject: [OmniOS-discuss] Installation of OmniOS on Intel NUC...
Message-ID: <026f01d04588$2ccbd8d0$86638a70$@siliconcloudinternational.com>

I'm working on a desktop-like installation of OmniOS on an Intel NUC
(http://www.intel.com/content/www/us/en/nuc/nuc-kit-d54250wykh.html).  I
have previously installed SmartOS on this NUC with no issues.  I built the
USB image, but when I boot the NUC, I am dropped into a grub shell with no
instructions.  Any thoughts on what I should be seeing upon installation
startup for OmniOS?  I'll get more into the description of my next phase of
installation & configuration once I get OmniOS installed.

 

Kind regards, Marc

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150210/4f3c0d6b/attachment-0001.html>

From danmcd at omniti.com  Thu Feb 12 01:06:42 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 11 Feb 2015 20:06:42 -0500
Subject: [OmniOS-discuss] Small update to UNZIP covers CVE 2014-9636
In-Reply-To: <EFCB2DF0-6A3D-4E3C-9888-3C5775205AC2@omniti.com>
References: <EFCB2DF0-6A3D-4E3C-9888-3C5775205AC2@omniti.com>
Message-ID: <9FF847D2-0D79-4B12-A2D3-BBBD30E9FC7A@omniti.com>


> On Feb 9, 2015, at 9:22 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> Thanks to Marissa Murphy for patching this.  It's now available for "pkg update" on all supported repos:  r151006/LTS, r151010/last-Stable, r151012/Stable, and r151013/bloody.
> 
> Won't require a reboot, nor service restarts since these are just command replacements.

You'll have to re-update unzip again, as the upstream (oss-security mailing list) discussion found a problem with the first fix.

Sorry about that!
Dan


From tim at multitalents.net  Thu Feb 12 05:32:26 2015
From: tim at multitalents.net (Tim Rice)
Date: Wed, 11 Feb 2015 21:32:26 -0800 (PST)
Subject: [OmniOS-discuss] Small update to UNZIP covers CVE 2014-9636
In-Reply-To: <9FF847D2-0D79-4B12-A2D3-BBBD30E9FC7A@omniti.com>
References: <EFCB2DF0-6A3D-4E3C-9888-3C5775205AC2@omniti.com>
	<9FF847D2-0D79-4B12-A2D3-BBBD30E9FC7A@omniti.com>
Message-ID: <alpine.UW2.2.11.1502112128440.2226@server01.int.multitalents.net>


On Wed, 11 Feb 2015, Dan McDonald wrote:

| 
| > On Feb 9, 2015, at 9:22 PM, Dan McDonald <danmcd at omniti.com> wrote:
| > 
| > Thanks to Marissa Murphy for patching this.  It's now available for "pkg update" on all supported repos:  r151006/LTS, r151010/last-Stable, r151012/Stable, and r151013/bloody.
| > 
| > Won't require a reboot, nor service restarts since these are just command replacements.
| 
| You'll have to re-update unzip again, as the upstream (oss-security mailing list) discussion found a problem with the first fix.

And possibly once more. Have a look at CVE-2014-8139 discussed at
https://bugzilla.redhat.com/show_bug.cgi?id=1174844

| Sorry about that!

These things happen.

| Dan
| 

-- 
Tim Rice				Multitalents	(707) 456-1146
tim at multitalents.net



From Joseph.Cardenuto at gdit.com  Thu Feb 12 22:12:46 2015
From: Joseph.Cardenuto at gdit.com (Cardenuto, Joseph P)
Date: Thu, 12 Feb 2015 22:12:46 +0000
Subject: [OmniOS-discuss] omnios problem with pc card and sd card
Message-ID: <77AE85424D6B1C4FA5F25AB148B55E694F84A479@HQ200-EXMBX03.ad.local>

Good Evening

            I am reaching out to see if anyone can help me with a problem. I am having a problem with OmniOS and I am hoping someone can help me solve the issue.
I installed OmniOS on a Getac 300G ruggedized laptop. The Machines contain an "Intel Core i7 vpro" and a Ricoh R5U242 (for the PC and SD card).
We are having problems getting the PC and SD cards to be recognized.

The problem we are having is twofold.

First the SD card is not even recognized so no interrupts are triggered.

Second when I insert the pc card which is a 250 mb drive I receive an error from pcmcia_map_reg.
The following is a dtrace snippet from inserting the card. From viewing the code the highlighted section is where it appears the problem is.

I would appreciate any help you can provide in solving this problem.


  3                          -> SocketServices [957780418290] pcmcia:SocketServices:entry
  3                            -> SSSetWindow [957780420042] pcmcia:SSSetWindow:entry
  3                              -> pcic_set_window [957780421870] pcic:pcic_set_window:entry
  3                                -> pcmcia_alloc_mem [957780424048] pcmcia:pcmcia_alloc_mem:entry
  3                                  -> pcmcia_ra_alloc [957780425947] pcmcia:pcmcia_ra_alloc:entry
  3                                    -> ddi_prop_free [957780432267] genunix:ddi_prop_free:entry
  3                                    <- ddi_prop_free [957780434818] genunix:ddi_prop_free:return
  3                                  <- pcmcia_ra_alloc [957780436735] pcmcia:pcmcia_ra_alloc:return
  3                                <- pcmcia_alloc_mem [957780438426] pcmcia:pcmcia_alloc_mem:return
  3                                -> pcmcia_map_reg [957780440700] pcmcia:pcmcia_map_reg:entry
  3                                  -> pcmcia_cons_regspec [957780442737] pcmcia:pcmcia_cons_regspec:entry
  3                                  <- pcmcia_cons_regspec [957780445271] pcmcia:pcmcia_cons_regspec:return
  3                                  -> ddi_map [957780448254] genunix:ddi_map:entry
  3                                    -> pcieb_bus_map [957780451272] pcieb:pcieb_bus_map:entry
  3                                      -> pci_set_prot_scan [957780457548] npe:pci_set_prot_scan:entry
  3                                      <- pci_set_prot_scan [957780459853] npe:pci_set_prot_scan:return
  3                                      -> npe_bus_map [957780454444] npe:npe_bus_map:entry
  3                                        -> pci_common_get_reg_prop [957780457548] npe:pci_common_get_reg_prop:entry
  3                                        <- pci_common_get_reg_prop [957780459853] npe:pci_common_get_reg_prop:return
  3                                        -> npe_is_mmcfg_supported [957780462061] npe:npe_is_mmcfg_supported:entry
  3                                          -> npe_child_is_pci [957780466185] npe:npe_child_is_pci:entry
  3                                            -> ddi_prop_free [957780468052] genunix:ddi_prop_free:entry
  3                                            <- ddi_prop_free [957780469435] genunix:ddi_prop_free:return
  3                                            -> ddi_prop_free [957780471001] genunix:ddi_prop_free:entry
  3                                            <- ddi_prop_free [957780472417] genunix:ddi_prop_free:return
  3                                          <- npe_child_is_pci [957780474081] npe:npe_child_is_pci:return
  3                                        <- npe_is_mmcfg_supported [957780475909] npe:npe_is_mmcfg_supported:return
  3                                        -> ddi_prop_lookup_int64_array [957780457548] genunix:ddi_prop_lookup_int64_array:entry
  3                                        <- ddi_prop_lookup_int64_array [957780459853] genunix:ddi_prop_lookup_int64_array:return
  3                                        -> ddi_prop_free [957780471001] genunix:ddi_prop_free:entry
  3                                        <- ddi_prop_free [957780472417] genunix:ddi_prop_free:return
  3                                      <- npe_bus_map [957780479691] npe:npe_bus_map:return
  3                                    <- pcieb_bus_map [957780481368] pcieb:pcieb_bus_map:return
  3                                  <- ddi_map [957780483673] genunix:ddi_map:return
  3                                <- pcmcia_map_reg [957780485827] pcmcia:pcmcia_map_reg:return
  3                                -> pcmcia_free_mem [957780501512] pcmcia:pcmcia_free_mem:entry
  3                                  -> pcmcia_ra_free [957780513753] pcmcia:pcmcia_ra_free:entry
  3                                    -> ddi_prop_free [957780516005] genunix:ddi_prop_free:entry
  3                                    <- ddi_prop_free [957780517440] genunix:ddi_prop_free:return
  3                                  <- pcmcia_ra_free [957780519082] pcmcia:pcmcia_ra_free:return
  3                                <- pcmcia_free_mem [957780520734] pcmcia:pcmcia_free_mem:return
  3                              <- pcic_set_window [957780522347] pcic:pcic_set_window:return
  3                            <- SSSetWindow [957780524357] pcmcia:SSSetWindow:return
  3                          <- SocketServices [957780525962] pcmcia:SocketServices:return

Thank you in advance for any help you can provide,
Joe
__________________________________________

Joseph (Joe) P. Cardenuto
Principal Engineer -- Software

General Dynamics Information Technology
55 Dodge Road, Suite 120
Getzville, NY  14068

Tel: (716) 243-4052
Fax: (716) 243-4011
Email: joseph.cardenuto at gdit.com<mailto:joseph.cardenuto at gdit.com>

[cid:image001.gif at 01CE41DA.90630ED0]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150212/6a91ec54/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 2766 bytes
Desc: image001.gif
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150212/6a91ec54/attachment-0001.gif>

From omnios.discuss at virtualunix.eu  Fri Feb 13 09:56:23 2015
From: omnios.discuss at virtualunix.eu (Michael)
Date: Fri, 13 Feb 2015 09:56:23 +0000
Subject: [OmniOS-discuss] Zone Discrepancy
Message-ID: <a49dc57d3d241f02c9940734cb266117@virtualunix.eu>

I've run into this issue a couple of times under different 
circumstances. In my original setup I used a template zone and then 
created a number of zones that were cloned from it. Somewhere along the 
line one of the clones would become promoted instead, which messed 
things up somewhat if I wanted to create a new clone from the template 
zone.

After a hardware upgrade I decided to use full installs for each zone 
instead of cloning to avoid the above situation, recently I saw the same 
thing happen again, ie after initial install
zones/tmpl0   ~500MB
zones/<zone>0 ~500MB
zones/<zone>1 ~500MB

and now
zones/tmpl0   ~30MB
zones/<zone>0 ~500MB
zones/<zone>1 ~1000MB

None of the zones had been detached when I noticed this, 
detaching/attaching/updating didn't change anything. While it's not such 
a big deal in this scenario because I'm not cloning (at this time) I 
can't see why this is occurring in the first place. Is this something 
anyone else is experiencing or any pointers in how to remedy this?

Michael

From cj.keist at colostate.edu  Mon Feb 16 23:44:59 2015
From: cj.keist at colostate.edu (CJ Keist)
Date: Mon, 16 Feb 2015 16:44:59 -0700
Subject: [OmniOS-discuss] 3ware LSI 9750-16i4e
Message-ID: <54E280FB.3010805@colostate.edu>

Hello,
    Looking for anyone that might have used this raid controller card? 
LSI web page states the raid controller is supported for OpenSolaris, 
but I'm not having any luck getting OmniOS 151012 to see any defined 
drives from the raid card.

I've tried to install driver from LSI but get the following error:
root at projects1:/root/install/intel/Solaris11# pkgadd -d .

The following packages are available:
   1  mrsas     LSI MegaRAID SAS 2.0 HBA driver
                (i386) 6.602.03.00

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]:

Processing package instance <mrsas> from </root/install/intel/Solaris11>

LSI MegaRAID SAS 2.0 HBA driver(i386) 6.602.03.00
Copyright (c) 2008-2009, LSI Logic Corporation.
All rights reserved.

## Executing checkinstall script.
pkgparam: ERROR: unable to locate parameter information for "mrsas"
A previous instance of mrsas driver  found in the system.
Use 'pkgrm mrsas' to remove the previous mrsas package and then do a pkgadd.
checkinstall script suspends

Installation of <mrsas> was suspended (administration).
No changes were made to the system.

Try to remove the SUNWmrsas package gets me this:

root at projects1:/root# pkgrm SUNWmrsas

The following package is currently installed:
    SUNWmrsas  LSI MegaRAID SAS2.0 HBA Driver
               (i386) 11.11,REV=2009.11.11

Do you want to remove this package? [y,n,?,q] y
pkgrm: ERROR: unable to change current working directory to 
</var/sadm/pkg/SUNWmrsas/install>

Removal of <SUNWmrsas> failed (internal error).
No changes were made to the system.


Anything I'm missing here?

-- 
C. J. Keist                     Email: cj.keist at colostate.edu
Systems Group Manager           Solaris 10 OS (SAI)
Engineering Network Services    Phone: 970-491-0630
College of Engineering, CSU     Fax:   970-491-5569
Ft. Collins, CO 80523-1301

All I want is a chance to prove 'Money can't buy happiness'

From danmcd at omniti.com  Tue Feb 17 01:30:30 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 16 Feb 2015 20:30:30 -0500
Subject: [OmniOS-discuss] 3ware LSI 9750-16i4e
In-Reply-To: <54E280FB.3010805@colostate.edu>
References: <54E280FB.3010805@colostate.edu>
Message-ID: <62EC32E5-67D7-47A8-8E18-27E16466F572@omniti.com>


> On Feb 16, 2015, at 6:44 PM, CJ Keist <cj.keist at colostate.edu> wrote:
> 
> Hello,
>   Looking for anyone that might have used this raid controller card? LSI web page states the raid controller is supported for OpenSolaris, but I'm not having any luck getting OmniOS 151012 to see any defined drives from the raid card.

First off... illumos != Solaris.  Stop thinking that way.  Oracle burned that bridge.

Second off, the LSI 2108 chipset SHOULD be supported.  I'd be curious for you to share "prtconf | grep 1000," so I can see the PCIe device identifier. 2108 is an older chipset.

> Anything I'm missing here?

Yes.  You shouldn't be getting a hardware RAID card for a ZFS filesystem.  This card is based on the LSI 2108 chipset, i.e. HW-RAID.  The ONLY reason I can see you wanting such a card is because of the 16i4e (16 + 4).

Ideally, you want something based on either the 2008, 2308, or as of r151012, the 3008 chipset.  Try something on this page:

http://www.lsi.com/products/host-bus-adapters/pages/default.aspx#tab/product-family-tab-1

Or

http://www.lsi.com/products/host-bus-adapters/pages/default.aspx#tab/product-family-tab-2

Hope this helps,
Dan


From jimklimov at cos.ru  Tue Feb 17 07:19:01 2015
From: jimklimov at cos.ru (Jim Klimov)
Date: Tue, 17 Feb 2015 08:19:01 +0100
Subject: [OmniOS-discuss] 3ware LSI 9750-16i4e
In-Reply-To: <54E280FB.3010805@colostate.edu>
References: <54E280FB.3010805@colostate.edu>
Message-ID: <D447EF2F-27F0-48CB-ADA1-0C0474B8848C@cos.ru>

17 ??????? 2015??. 0:44:59 CET, CJ Keist <cj.keist at colostate.edu> ?????:
>Hello,
>    Looking for anyone that might have used this raid controller card? 
>LSI web page states the raid controller is supported for OpenSolaris, 
>but I'm not having any luck getting OmniOS 151012 to see any defined 
>drives from the raid card.
>
>I've tried to install driver from LSI but get the following error:
>root at projects1:/root/install/intel/Solaris11# pkgadd -d .
>
>The following packages are available:
>   1  mrsas     LSI MegaRAID SAS 2.0 HBA driver
>                (i386) 6.602.03.00
>
>Select package(s) you wish to process (or 'all' to process
>all packages). (default: all) [?,??,q]:
>
>Processing package instance <mrsas> from
></root/install/intel/Solaris11>
>
>LSI MegaRAID SAS 2.0 HBA driver(i386) 6.602.03.00
>Copyright (c) 2008-2009, LSI Logic Corporation.
>All rights reserved.
>
>## Executing checkinstall script.
>pkgparam: ERROR: unable to locate parameter information for "mrsas"
>A previous instance of mrsas driver  found in the system.
>Use 'pkgrm mrsas' to remove the previous mrsas package and then do a
>pkgadd.
>checkinstall script suspends
>
>Installation of <mrsas> was suspended (administration).
>No changes were made to the system.
>
>Try to remove the SUNWmrsas package gets me this:
>
>root at projects1:/root# pkgrm SUNWmrsas
>
>The following package is currently installed:
>    SUNWmrsas  LSI MegaRAID SAS2.0 HBA Driver
>               (i386) 11.11,REV=2009.11.11
>
>Do you want to remove this package? [y,n,?,q] y
>pkgrm: ERROR: unable to change current working directory to 
></var/sadm/pkg/SUNWmrsas/install>
>
>Removal of <SUNWmrsas> failed (internal error).
>No changes were made to the system.
>
>
>Anything I'm missing here?

A couple of things pop out. 

One is that failures are usually programmatic, i.e. due to tests or other actions in a pre- or post- install/remove scripts. You can inspect those (an SVR4 package is a glorified cpio archive containing the metadata files and ultimate packaged contents - maybe inlaid in a none.(gz,7z,...) archive. More scientifically, you can use pkgtrans to pack\unpack the package delivery formats, such as a single file, a directory, etc. With some handywork (also fixing pkginfo with its cksum-based checksums), you can instrument the pre\post scripts with set -x to trace and repackage, to see what does not work why during an actual installation. 

It may be that uninstallation fails due to poor legacy dependencies, e.g. if SUNWcs (~kernel) 'depends' on some driver packages like this storage, so you can't remove the old one, and a new driver package may consider itself to be in conflict with the old one. I'm enroute away from a computer now so can't say if this is still the case with illumos metadata. But if this is indeed the roadblock, you might need some pointier surgical tools or a larger hammer to add/remove the driver files and/or the packagjng system's perception of the package.

Also look in dmesg - probably some driver modules fail to attach or detach, although generally installation of a dormant driver into an OS image should not depend on actual presence of a device.

Oh, also you can clone and mount an alternate BE and use the pkgadd/pkgrm with -R to manipulate this other OS image, maybe then the hardware related checks would be more relaxed.

Finally, the 3wares i knew (before they were bought by lsi) were using a chip and firmware of their own. I did not use the later products, but if they are not just a rebrand of mainstream lsi with another known brand - double-check that you need an lsi/megaraid driver or some other after all.

HTH,
Jim Klimov
--
Typos courtesy of K-9 Mail on my Samsung Android

From b3niup at gmail.com  Tue Feb 17 14:31:38 2015
From: b3niup at gmail.com (=?UTF-8?Q?Benedykt_Przyby=C5=82o?=)
Date: Tue, 17 Feb 2015 15:31:38 +0100
Subject: [OmniOS-discuss] L2ARC issue - 16.0E SSD?
Message-ID: <CAJ9qmG=kEpO=rQCCeGWe2NniGbKYZDW1z=hmMt8T0hqPP5MASA@mail.gmail.com>

Hello,

I am having problems with L2ARC SSD drives since upgrade to omnios-10b9c79.
After filling cache up l2arc size changes to 16.0E and zpool iostat -v
shows that cache is still growing (at this moment it is using 2.60T out of
16.0E available space).
After removing l2arc drive from pool and reattaching it problem disappears
until cache grows up again.
Problem affects two machines with similar setup (and my only two machines
that runs omnios-10b9c79).

For l2arc purpose I am using Intel SSD DC S3500 480GB SATA 2,5".
Few system details:

# uname -a
SunOS nb3 5.11 omnios-10b9c79 i86pc i386 i86pc

# zpool status | grep cache -A 1
        cache
          c1t55CD2E404B575172d0    ONLINE       0     0     0

# zpool iostat -v | grep cache -A 1
cache                          -      -      -      -      -      -
  c1t55CD2E404B575172d0    2.60T  16.0E    735    200  1.37M  2.73M

# kstat zfs::arcstats:*l2*
module: zfs                             instance: 0
name:   arcstats                        class:    misc
        evict_l2_cached                 29975692498432
        evict_l2_eligible               10552753603072
        evict_l2_ineligible             1821252098048
        l2_abort_lowmem                 1220
        l2_asize                        2856707439616
        l2_cksum_bad                    584529772
        l2_compress_failures            0
        l2_compress_successes           102235818
        l2_compress_zeros               0
        l2_evict_lock_retry             409
        l2_evict_reading                0
        l2_feeds                        1300435
        l2_free_on_write                11906910
        l2_hdr_size                     34884171088
        l2_hits                         765898065
        l2_io_error                     103979415
        l2_misses                       1910226514
        l2_read_bytes                   1495882560512
        l2_rw_clash                     535147
        l2_size                         3757422951424
        l2_write_bytes                  2984588264448
        l2_writes_done                  916469
        l2_writes_error                 0
        l2_writes_hdr_miss              65671
        l2_writes_sent                  916469

# fmadm faulty
# fmdump -eV
TIME                           CLASS
fmdump: warning: /var/fm/fmd/errlog is empty

# zpool get version DATA3-BACKUP
NAME          PROPERTY  VALUE    SOURCE
DATA3-BACKUP  version   28       local


I found similar issue in freenas: https://bugs.freenas.org/issues/6239
With such patch as solution:
https://bugs.freenas.org/projects/freenas/repository/trueos/revisions/6ec48ebf5a1596ec7d2732e891fce3f116105ae5/diff/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c


I would like to know if this will be fixed in a stable release soon or if
there will be any patch that I will be able to apply to my current
environment to fix this?

Best regards,
Benedykt Przyby?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150217/17969805/attachment.html>

From cj.keist at colostate.edu  Tue Feb 17 16:20:50 2015
From: cj.keist at colostate.edu (CJ Keist)
Date: Tue, 17 Feb 2015 09:20:50 -0700
Subject: [OmniOS-discuss] 3ware LSI 9750-16i4e
In-Reply-To: <62EC32E5-67D7-47A8-8E18-27E16466F572@omniti.com>
References: <54E280FB.3010805@colostate.edu>
	<62EC32E5-67D7-47A8-8E18-27E16466F572@omniti.com>
Message-ID: <54E36A62.8040406@colostate.edu>

Thank you.
    Yes, realize that Illumos is not Solaris. But I know that Illumos is 
a branch from OpenSolaris after Oracle decided they didn't want to share 
the same sandbox with everyone else.  I thought it safe to purchase this 
raid controller.  This controller does support JBOD in the that you can 
define each hard disk as a single drive.

Just for information. I was able to remove the SUNWmrsas package.
1. mkdir /var/sadm/pkg/SUNWmrsas/install
2. edit /var/sadm/pkg/SUNWmrsas/pkginfo
    Add "CLASSES=none".
Then pkgrm worked.
3. touch /reconfigure and reboot.

But the LSI driver would not load, still thought mr_sas driver was 
present. So modified the checkinstall script to pass.  But then got error:
pkgadd: ERROR: packaging file 
</root/install/intel/Solaris11/mrsas/install/checkinstall> is corrupt
     file size <1391> expected <1071> actual
     file cksum <47879> expected <23422> actual

Edit pkgmap file and changed line:
1 i checkinstall 1391 47879 1384458730
to
1 i checkinstall 1071 23422 1384458730

That got the LSI drive installed. Again "touch /reconfigure" and reboot.

But even after all that no luck in seeing the defined drives.  modinfo 
didn't show any sas driver running, So did:
modload -p drv/mr_sas

modinfo | grep sas
255 fffffffff8125000  43da0 179   1  mr_sas (6.602.03.00)
devfsadm

Still nothing.  Looks like I'll purchase a different raid card.  Any 
recommendations?
I have system with Intel based mother board: E99552-510
16 4TB internal drives plus a SAS expansion with another 16 4TB hard drives.




On 2/16/15 6:30 PM, Dan McDonald wrote:
>
>> On Feb 16, 2015, at 6:44 PM, CJ Keist <cj.keist at colostate.edu> wrote:
>>
>> Hello,
>>    Looking for anyone that might have used this raid controller card? LSI web page states the raid controller is supported for OpenSolaris, but I'm not having any luck getting OmniOS 151012 to see any defined drives from the raid card.
>
> First off... illumos != Solaris.  Stop thinking that way.  Oracle burned that bridge.
>
> Second off, the LSI 2108 chipset SHOULD be supported.  I'd be curious for you to share "prtconf | grep 1000," so I can see the PCIe device identifier. 2108 is an older chipset.
>
>> Anything I'm missing here?
>
> Yes.  You shouldn't be getting a hardware RAID card for a ZFS filesystem.  This card is based on the LSI 2108 chipset, i.e. HW-RAID.  The ONLY reason I can see you wanting such a card is because of the 16i4e (16 + 4).
>
> Ideally, you want something based on either the 2008, 2308, or as of r151012, the 3008 chipset.  Try something on this page:
>
> http://www.lsi.com/products/host-bus-adapters/pages/default.aspx#tab/product-family-tab-1
>
> Or
>
> http://www.lsi.com/products/host-bus-adapters/pages/default.aspx#tab/product-family-tab-2
>
> Hope this helps,
> Dan
>

-- 
C. J. Keist                     Email: cj.keist at colostate.edu
Systems Group Manager           Solaris 10 OS (SAI)
Engineering Network Services    Phone: 970-491-0630
College of Engineering, CSU     Fax:   970-491-5569
Ft. Collins, CO 80523-1301

All I want is a chance to prove 'Money can't buy happiness'

From danmcd at omniti.com  Tue Feb 17 16:28:46 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 17 Feb 2015 11:28:46 -0500
Subject: [OmniOS-discuss] 3ware LSI 9750-16i4e
In-Reply-To: <54E36A62.8040406@colostate.edu>
References: <54E280FB.3010805@colostate.edu>
	<62EC32E5-67D7-47A8-8E18-27E16466F572@omniti.com>
	<54E36A62.8040406@colostate.edu>
Message-ID: <E70B4493-CC99-4AD1-8CD6-7CF060B5A703@omniti.com>

I don't see a PCI ID for your board.

It is POSSIBLE the BUILT-IN mr_sas on OmniOS will do what you want, but I *need* to see that PCI ID for your board.

Dan


From omnios at citrus-it.net  Wed Feb 18 13:41:30 2015
From: omnios at citrus-it.net (Andy)
Date: Wed, 18 Feb 2015 13:41:30 +0000 (GMT)
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
Message-ID: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>


Hi,

We have a number of Dell PE1950 servers running OmniOS which are in need
of replacement. Management here have always used Dell and have suggested
the R730xd platform (as we want more disk slots in the new servers).

I've read some useful posts on this mailing list about the Dell R730xd -
specifically that if we ask Dell to configure it for Nexenta then it will
come with an LSI HBA and Intel NICs but I haven't seen any further posts
saying whether anyone has had any success getting one of these up and
running. I got the impression from the posts that it wasn't a great idea
and Supermicro is a better option.

I'd prefer to run Supermicro but might have some problems convincing
those with the purse strings. Is anyone running, or can anyone recommend,
a Supermicro server roughly equivalent to the Del R730xd, or give me an
idea of what chipsets/HBAs etc. to choose or avoid for OmniOS?

Any help appreciated,

Thanks

Andy

-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From illumos at cucumber.demon.co.uk  Wed Feb 18 14:09:56 2015
From: illumos at cucumber.demon.co.uk (Andrew Gabriel)
Date: Wed, 18 Feb 2015 14:09:56 +0000
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
Message-ID: <54E49D34.9090709@cucumber.demon.co.uk>

In customers I've dealt with, if there isn't a binding data center 
policy in place for one or the other, it's usually a trade-off between 
TCO, and the differing field service offerings with respect to mean time 
to repair (which can be very location-specific). This can also be 
influenced by the ability of your own data center staff to 
diagnose/repair hardware faults and your own spares holding, and of 
course the availability requirements of your systems.

Andy wrote:
> Hi,
>
> We have a number of Dell PE1950 servers running OmniOS which are in need
> of replacement. Management here have always used Dell and have suggested
> the R730xd platform (as we want more disk slots in the new servers).
>
> I've read some useful posts on this mailing list about the Dell R730xd -
> specifically that if we ask Dell to configure it for Nexenta then it will
> come with an LSI HBA and Intel NICs but I haven't seen any further posts
> saying whether anyone has had any success getting one of these up and
> running. I got the impression from the posts that it wasn't a great idea
> and Supermicro is a better option.
>
> I'd prefer to run Supermicro but might have some problems convincing
> those with the purse strings. Is anyone running, or can anyone recommend,
> a Supermicro server roughly equivalent to the Del R730xd, or give me an
> idea of what chipsets/HBAs etc. to choose or avoid for OmniOS?
>
> Any help appreciated,
>
> Thanks
>
> Andy
>
>   


From doug at will.to  Wed Feb 18 14:07:36 2015
From: doug at will.to (Doug Hughes)
Date: Wed, 18 Feb 2015 09:07:36 -0500
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
Message-ID: <71069fcd-7674-4e3f-8ded-a630df3258c7.maildroid@localhost>

The nice thing about Supermicro is that they have much for platform variation than just about anybody else. However, if you need the r730 with the 1.8 disks, SM does not have an option for that. You can, however, easily get 24 2.5" disks in the SC216 (or equivalent) platform, and they have several platforms in 1U and 2U that can accommodate a wide variety of 3.5"disk choices. Which are you looking for? You probably want to give an integrator your specs and let them suggest one. There are many available.

Berkeley Communications (my favorite)
Microway
Colfax
many more.

You also might look at PSSC labs to see if one of their items fits.


Sent from my android device.

-----Original Message-----
From: Andy <omnios at citrus-it.net>
To: omnios-discuss at lists.omniti.com
Sent: Wed, 18 Feb 2015 8:51
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..


Hi,

We have a number of Dell PE1950 servers running OmniOS which are in need
of replacement. Management here have always used Dell and have suggested
the R730xd platform (as we want more disk slots in the new servers).

I've read some useful posts on this mailing list about the Dell R730xd -
specifically that if we ask Dell to configure it for Nexenta then it will
come with an LSI HBA and Intel NICs but I haven't seen any further posts
saying whether anyone has had any success getting one of these up and
running. I got the impression from the posts that it wasn't a great idea
and Supermicro is a better option.

I'd prefer to run Supermicro but might have some problems convincing
those with the purse strings. Is anyone running, or can anyone recommend,
a Supermicro server roughly equivalent to the Del R730xd, or give me an
idea of what chipsets/HBAs etc. to choose or avoid for OmniOS?

Any help appreciated,

Thanks

Andy

-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150218/ba536c44/attachment.html>

From danmcd at omniti.com  Wed Feb 18 15:34:18 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 18 Feb 2015 10:34:18 -0500
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
Message-ID: <94F01EF5-4FF3-4737-98CE-097D81D6ED80@omniti.com>


> On Feb 18, 2015, at 8:41 AM, Andy <omnios at citrus-it.net> wrote:
> 
> I've read some useful posts on this mailing list about the Dell R730xd -
> specifically that if we ask Dell to configure it for Nexenta then it will
> come with an LSI HBA and Intel NICs but I haven't seen any further posts
> saying whether anyone has had any success getting one of these up and
> running. I got the impression from the posts that it wasn't a great idea
> and Supermicro is a better option.

The key is to get an HBA illumos likes.  The *default* Dell HBAs aren't all that great, and stick you with HW-RAID.

I've not heard success stories with "Nexenta Configured" + OmniOS, but that's because every Nexenta configuration was used to... well... run NexentaStor.  :)

I know lots of illumos shops run SuperMicro with success.

Dan


From lkateley at kateley.com  Wed Feb 18 15:49:15 2015
From: lkateley at kateley.com (Linda Kateley)
Date: Wed, 18 Feb 2015 09:49:15 -0600
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <94F01EF5-4FF3-4737-98CE-097D81D6ED80@omniti.com>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
	<94F01EF5-4FF3-4737-98CE-097D81D6ED80@omniti.com>
Message-ID: <54E4B47B.1040205@kateley.com>

Just some anecdotal info... sun had a very nice partnership with dell.. 
back in the day, if you searched for dell in the hardware compat list 
you would show thousand of entries.. Dell now has a guy who sits on 
nexenta's board, this fact might have some impact in what they recommend.

On the other hand, most if not all of the companies selling openzfs 
appliances are running supermicro or similar gear...Except for 
compellent which definitely runs dell :)

I agree with dan though, the hba is the most important, as is nics.

All this being said, I would personally prefer that dell r730 if i had a 
choice...

lk

On 2/18/15 9:34 AM, Dan McDonald wrote:
>> On Feb 18, 2015, at 8:41 AM, Andy <omnios at citrus-it.net> wrote:
>>
>> I've read some useful posts on this mailing list about the Dell R730xd -
>> specifically that if we ask Dell to configure it for Nexenta then it will
>> come with an LSI HBA and Intel NICs but I haven't seen any further posts
>> saying whether anyone has had any success getting one of these up and
>> running. I got the impression from the posts that it wasn't a great idea
>> and Supermicro is a better option.
> The key is to get an HBA illumos likes.  The *default* Dell HBAs aren't all that great, and stick you with HW-RAID.
>
> I've not heard success stories with "Nexenta Configured" + OmniOS, but that's because every Nexenta configuration was used to... well... run NexentaStor.  :)
>
> I know lots of illumos shops run SuperMicro with success.
>
> Dan
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From rt at steait.net  Wed Feb 18 20:04:18 2015
From: rt at steait.net (Rune Tipsmark)
Date: Wed, 18 Feb 2015 20:04:18 +0000
Subject: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
Message-ID: <1424289858252.20555@steait.net>

hi all,



I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII

it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever.



I have sync=always and disabled write back cache on my volume based LU's.



Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least.



How do I ensure 100% that all writes always goes to my Slog devices - no exceptions.



br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150218/033ac43e/attachment-0001.html>

From chip at innovates.com  Wed Feb 18 20:26:55 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 18 Feb 2015 14:26:55 -0600
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
Message-ID: <CALeZrrS03JpYQ799jV_L77SzRHmyo=Q48yB_Yf0m1aajduJYyw@mail.gmail.com>

On Wed, Feb 18, 2015 at 7:41 AM, Andy <omnios at citrus-it.net> wrote:

>
>
> I'd prefer to run Supermicro but might have some problems convincing
> those with the purse strings. Is anyone running, or can anyone recommend,
> a Supermicro server roughly equivalent to the Del R730xd, or give me an
> idea of what chipsets/HBAs etc. to choose or avoid for OmniOS?
>
>
I've been quite happy with the 6028U-TR4+.   It was the first 2U Ultra
server Supermicro was shipping.   Any of the 2U Ultra's would be a good
choice.   What I like about these in particular is lots of PCIe slots for
HBAs and 10GB NICs.   If your running RJ45 10GB they have versions with
10GB built in.

If it were available at the time I would have went with the 2028U, since it
has all 2 1/2 drives.   The only drives I've put in the SATA side has been
SSDs for boot and L2ARC.   With an internal HBA you could load it up with
24 drives.  I think with the on board SATA you are limited to 10 drives.

-Chip



> Any help appreciated,
>
> Thanks
>
> Andy
>
> --
> Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
> Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
> Registered in England and Wales | Company number 4899123
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150218/1d7acc8c/attachment.html>

From martin at dojcak.sk  Wed Feb 18 22:34:08 2015
From: martin at dojcak.sk (Martin Dojcak)
Date: Wed, 18 Feb 2015 23:34:08 +0100
Subject: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
Message-ID: <gkaofi7243j176qa2tl3cic0.1424296594181@email.android.com>

Hi,

zil_slog_limit from illumos-gate source code is really 1024*1024 bytes for one write operation. There is nice self explain thread on zfs on linux github (https://github.com/zfsonlinux/zfs/issues/1012) when a write should be logged in indirect mode or in immediate mode. Application rarely use write > 1MB. In the most cases 1MB is enaugh, but it would be nice to have a option to configure zil slog limit (like zfs on linux module).


--
Martin Doj??k

mail: martin at dojcak.sk
jabber: martindojcak at jabbim.sk

pgp: http://pgp.dojcak.sk/



<div>-------- P?vodn? spr?va --------</div><div>Od: Rune Tipsmark <rt at steait.net> </div><div>D?tum:18. 02. 2015  21:04  (GMT+01:00) </div><div>Komu: omnios-discuss at lists.omniti.com </div><div>Predmet: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog </div><div>
</div>hi all,
 
I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII
it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever.
 
I have sync=always and disabled write back cache on my volume based LU's.
 
Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least.
 
How do I ensure 100% that all writes always goes to my Slog devices - no exceptions.
 
br,
Rune  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150218/4ef0f687/attachment.html>

From richard.elling at richardelling.com  Thu Feb 19 00:27:00 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Wed, 18 Feb 2015 16:27:00 -0800
Subject: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
In-Reply-To: <1424289858252.20555@steait.net>
References: <1424289858252.20555@steait.net>
Message-ID: <9B0B4A05-73D1-4124-BEF5-8CEE5E37D0F3@richardelling.com>


> On Feb 18, 2015, at 12:04 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi all,
>  
> I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII <http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII>
> it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html <http://nex7.blogspot.ca/2013/04/zfs-intent-log.html> but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever.
> 

"never ever want anything to go to main pool" is not feasible. The ZIL is a ZFS Intent Log
http://en.wikipedia.org/wiki/Intent_log <http://en.wikipedia.org/wiki/Intent_log> and, unless you overwrite prior to txg commit, everything
ends up in the main pool.

>  
> I have sync=always and disabled write back cache on my volume based LU's.
>  
> Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least.
> 

Ultimately, the pool must be able to sustain the workload, or it will have to throttle.

The comment for zil_slog_limit is concise:
/*
 * Use the slog as long as the logbias is 'latency' and the current commit size
 * doesn't exceed the limit or the total list size doesn't exceed its limit.
 * Limit checking is disabled by setting zil_slog_limit to UINT64_MAX.
 */
uint64_t zil_slog_limit = (1024 * 1024);
uint64_t zil_slog_list_limit = (1024 * 1024 * 200);

and you can change this on the fly using mdb to experiment.

>  
> How do I ensure 100% that all writes always goes to my Slog devices - no exceptions.
> 

The question really isn't "how" the question is "why"? Now that you know what an 
Intent Log is, and how the performance of the pool is your ultimate limit, perhaps you
can explain what you are really trying to accomplish?
 -- richard

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150218/35a232a2/attachment.html>

From rt at steait.net  Thu Feb 19 02:09:36 2015
From: rt at steait.net (Rune Tipsmark)
Date: Thu, 19 Feb 2015 02:09:36 +0000
Subject: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
In-Reply-To: <9B0B4A05-73D1-4124-BEF5-8CEE5E37D0F3@richardelling.com>
References: <1424289858252.20555@steait.net>,
	<9B0B4A05-73D1-4124-BEF5-8CEE5E37D0F3@richardelling.com>
Message-ID: <1424311776360.64422@steait.net>

________________________________
From: Richard Elling <richard.elling at richardelling.com>
Sent: Thursday, February 19, 2015 1:27 AM
To: Rune Tipsmark
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog


On Feb 18, 2015, at 12:04 PM, Rune Tipsmark <rt at steait.net<mailto:rt at steait.net>> wrote:


hi all,



I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII

it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever.


"never ever want anything to go to main pool" is not feasible. The ZIL is a ZFS Intent Log
http://en.wikipedia.org/wiki/Intent_log and, unless you overwrite prior to txg commit, everything
ends up in the main pool.

>> yeah I know, I meant everything needs to go thru the ZIL before hitting the main pool...

I have sync=always and disabled write back cache on my volume based LU's.



Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least.


Ultimately, the pool must be able to sustain the workload, or it will have to throttle.

>> it should be OK to take in some hundreds MB/sec (11 SAS mirrors each can do ~150MB/Sec sequential)
The comment for zil_slog_limit is concise:
/*
 * Use the slog as long as the logbias is 'latency' and the current commit size
 * doesn't exceed the limit or the total list size doesn't exceed its limit.
 * Limit checking is disabled by setting zil_slog_limit to UINT64_MAX.
 */
uint64_t zil_slog_limit = (1024 * 1024);
uint64_t zil_slog_list_limit = (1024 * 1024 * 200);

and you can change this on the fly using mdb to experiment.

>> how do I do this? I could not find any property matching....



How do I ensure 100% that all writes always goes to my Slog devices - no exceptions.


The question really isn't "how" the question is "why"? Now that you know what an
Intent Log is, and how the performance of the pool is your ultimate limit, perhaps you
can explain what you are really trying to accomplish?

>> consistent fast write speeds at all times rather than yoyo write speeds... I get fine benchmarks but rather less fine file copy performance... and I never ever see disks being particular busy during file copy tests....
 -- richard

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150219/89c94321/attachment-0001.html>

From danmcd at omniti.com  Thu Feb 19 02:36:37 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 18 Feb 2015 21:36:37 -0500
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
Message-ID: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>

There will be only 1-3 more updates prior to the next OmniOS stable (and for this time, this stable is also Long-Term Support) release.  If you notice anything weird about the bloody bits, please let me know ASAP.  I've heard little/no complaints about the updates to pkg(5), so either you're very happy, or not using it.  :)

This update will only be reaching the repo.

* omnios-build master branch, revision f3d6d48

* Git to 2.3.0.

* UnZIP fixes.

* OpenJDK7 up to update 76, build 31.

* Microtasking libraries (/lib/libmtsk*) are back as a distinct package
  (system/library/mtsk) now that they are not part of the (now open-source)
  Math libraries.

* illumos-omnios master branch, revision cbf73e4 (last illumos-gate merge 336069c)

* Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.

Thanks,
Dan


From lars.nordin at photonet.se  Thu Feb 19 11:04:51 2015
From: lars.nordin at photonet.se (Lars Nordin)
Date: Thu, 19 Feb 2015 12:04:51 +0100
Subject: [OmniOS-discuss] no interface found installing kvm
Message-ID: <54E5C353.9070004@photonet.se>

Hi gurus!in
I try to to install pfsence in a kvm virtual machine. First in boot 
interfaces is found but later it prompts in vnc:
No interface found do you want to declare one?
And auto detect fails.

beginning of vm boot:

qemu-system-x86_64: -net 
vnic,vlan=0,name=net0,ifname=pfwan0,macaddr=2:8:20:93:f5:d9: vnic dhcp 
disabled

qemu-system-x86_64: -net 
vnic,vlan=1,name=net1,ifname=pflan0,macaddr=2:8:20:41:46:7d: vnic dhcp 
disabled


startscript:

#!/usr/bin/bash

# on omnios command is /usr/bin/qemu-system-x86_64

# configuration
NAME="PfSense2.13"
NUM=2
VNIC0=pfwan0
VNIC1=pflan0
#VNIC2=pftlout0
#VNIC3=pftlin0
#VNIC4=print0
#HDD=/dev/zvol/rdsk/mainpool/vm/pfsense/os
HDD=/dev/zvol/rdsk/daedaluspool/pfroot/os
#CD=/mainpool/nfs/iso/pf213.iso
CD=/mnt/pfSense-LiveCD-2.1.5-RELEASE-amd64.iso
MEM=2048

# don't change below here!

TLN=`expr 7000 + $NUM`
mac0=`dladm show-vnic -po macaddress $VNIC0`
mac1=`dladm show-vnic -po macaddress $VNIC1`
#mac2=`dladm show-vnic -po macaddress $VNIC2`
#mac3=`dladm show-vnic -po macaddress $VNIC3`
#mac4=`dladm show-vnic -po macaddress $VNIC4`

/usr/bin/qemu-system-x86_64 \
-name $NAME \
-boot cd \
-enable-kvm \
-vnc 0.0.0.0:$NUM \
-smp cores=1,threads=2,sockets=1 \
-m $MEM \
-no-hpet \
-localtime \
-drive file=$HDD,if=ide,index=0 \
-drive file=$CD,media=cdrom,if=ide,index=2 \
-net nic,vlan=0,name=net0,model=virtio,macaddr=$mac0 \
-net vnic,vlan=0,name=net0,ifname=$VNIC0,macaddr=$mac0 \
-net nic,vlan=1,name=net1,model=virtio,macaddr=$mac1 \
-net vnic,vlan=1,name=net1,ifname=$VNIC1,macaddr=$mac1 \
#-net nic,vlan=2,name=net2,model=virtio,macaddr=$mac2 \
#-net vnic,vlan=2,name=net2,ifname=$VNIC2,macaddr=$mac2 \
#-net nic,vlan=3,name=net3,model=virtio,macaddr=$mac3 \
#-net vnic,vlan=3,name=net3,ifname=$VNIC3,macaddr=$mac3 \
#-net nic,vlan=4,name=net4,model=virtio,macaddr=$mac4 \
#-net vnic,vlan=4,name=net4,ifname=$VNIC4,macaddr=$mac4 \
-vga std \
-cpu host \
-pidfile /pfroot/pids/$NAME.pid \
-monitor telnet:localhost:$TLN,server,nowait,nodelay \
-daemonize

if [ $? -gt 0 ]; then
     echo "Failed to start VM"
     exit
fi

port=`expr 5900 + $NUM`
public_nic=$(dladm show-vnic|grep vnic0|awk '{print $2}')
public_ip=$(ifconfig $public_nic|grep inet|awk '{print $2}')

echo "Started VM: $NAME"
echo "VNC available at: host IP ${public_ip} port ${port}"
echo "QEMU Monitor, do: # telnet localhost $TLN. Note: use Control ] to 
exit monitor before quit!
"
What am I missing?

/Lars Nordin

From danmcd at omniti.com  Thu Feb 19 16:16:02 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 19 Feb 2015 11:16:02 -0500
Subject: [OmniOS-discuss] no interface found installing kvm
In-Reply-To: <54E5C353.9070004@photonet.se>
References: <54E5C353.9070004@photonet.se>
Message-ID: <494DE363-39D3-4060-8838-42B6D5E26635@omniti.com>

See what "dladm show-link" and "dladm show-vnic" have to say.  It seems like they might not actually be there.

Dan


From johan.kragsterman at capvert.se  Thu Feb 19 18:08:31 2015
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 19 Feb 2015 19:08:31 +0100
Subject: [OmniOS-discuss] Ang: Re:  no interface found installing kvm
In-Reply-To: <494DE363-39D3-4060-8838-42B6D5E26635@omniti.com>
References: <494DE363-39D3-4060-8838-42B6D5E26635@omniti.com>,
	<54E5C353.9070004@photonet.se>
Message-ID: <OF6D2DEA58.E2A76A0D-ONC1257DF1.0063A851-C1257DF1.0063A854@inse.com>

Hi!

No need, Dan, this is a PfSense thing...needs to load virtio modules.

Johan

-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Lars Nordin <lars.nordin at photonet.se>
Fr?n: Dan McDonald 
S?nt av: "OmniOS-discuss" 
Datum: 2015-02-19 17:18
Kopia: omnios-discuss at lists.omniti.com
?rende: Re: [OmniOS-discuss] no interface found installing kvm

See what "dladm show-link" and "dladm show-vnic" have to say. ?It seems like they might not actually be there.

Dan

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss



From groups at tierarzt-mueller.de  Fri Feb 20 09:17:49 2015
From: groups at tierarzt-mueller.de (Alexander Lesle)
Date: Fri, 20 Feb 2015 10:17:49 +0100
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
Message-ID: <521254406.20150220101749@tierarzt-mueller.de>

Hello Dan McDonald and List,

do your remember my feature request?
,-----[  ]-----
|
| at Friday, 3. Okt. 2014 17:22 <Dan McDonald>
| at mid:A762C059-F687-42DE-A1F7-6CF2A7F5863B at omniti.com wrote:
| 
| > On Oct 3, 2014, at 9:29 AM, Alexander Lesle
| > <groups at tierarzt-mueller.de> wrote:
| 
| >> Hello Dan McDonald and List,
| >> 
| >> in my mind this was a great proposal what you done at Sep, 02.
| >> That's the way to build a great OmniOS near to the users.
| >> 
| >> Here my request for the next release:
| >> It would be convenient if open-vm-tools were installed in the new
| >> release. http://sourceforge.net/projects/open-vm-tools/files/
| 
| > Actually, this work has been done.  I dropped it on the ground:
| 
| >         https://github.com/omniti-labs/omnios-build/pull/39
| 
| > I will be merging this pull requests now in the master branch,
| > and seeing how OOod chews on it.
| 
| 
| > I know there's other work to be done for OmniOS-as-a-guest as well.
| 
| > Thanks,
| > DAn
|
`-------------------

Do you find the time to integrate open-vm-tools in the next stable
version?

Thanks.

On Februar, 19 2015, 03:36 <Dan McDonald> wrote in [1]:

> There will be only 1-3 more updates prior to the next OmniOS stable
> (and for this time, this stable is also Long-Term Support) release. 
> If you notice anything weird about the bloody bits, please let me
> know ASAP.  I've heard little/no complaints about the updates to
> pkg(5), so either you're very happy, or not using it.  :)

> This update will only be reaching the repo.

> * omnios-build master branch, revision f3d6d48

> * Git to 2.3.0.

> * UnZIP fixes.

> * OpenJDK7 up to update 76, build 31.

> * Microtasking libraries (/lib/libmtsk*) are back as a distinct package
>   (system/library/mtsk) now that they are not part of the (now open-source)
>   Math libraries.

> * illumos-omnios master branch, revision cbf73e4 (last illumos-gate merge 336069c)

> * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.

-- 
Best Regards
Alexander
Februar, 20 2015
........
[1] mid:F70C482D-68E8-43A2-BF73-86ADE4B63496 at omniti.com
........


From chip at innovates.com  Fri Feb 20 14:20:54 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Fri, 20 Feb 2015 08:20:54 -0600
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <521254406.20150220101749@tierarzt-mueller.de>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
Message-ID: <CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>

I will second that request as my OmniOS experiments always start as VMware
VMs.

-Chip

On Fri, Feb 20, 2015 at 3:17 AM, Alexander Lesle <groups at tierarzt-mueller.de
> wrote:

> Hello Dan McDonald and List,
>
> do your remember my feature request?
> ,-----[  ]-----
> |
> | at Friday, 3. Okt. 2014 17:22 <Dan McDonald>
> | at mid:A762C059-F687-42DE-A1F7-6CF2A7F5863B at omniti.com wrote:
> |
> | > On Oct 3, 2014, at 9:29 AM, Alexander Lesle
> | > <groups at tierarzt-mueller.de> wrote:
> |
> | >> Hello Dan McDonald and List,
> | >>
> | >> in my mind this was a great proposal what you done at Sep, 02.
> | >> That's the way to build a great OmniOS near to the users.
> | >>
> | >> Here my request for the next release:
> | >> It would be convenient if open-vm-tools were installed in the new
> | >> release. http://sourceforge.net/projects/open-vm-tools/files/
> |
> | > Actually, this work has been done.  I dropped it on the ground:
> |
> | >         https://github.com/omniti-labs/omnios-build/pull/39
> |
> | > I will be merging this pull requests now in the master branch,
> | > and seeing how OOod chews on it.
> |
> |
> | > I know there's other work to be done for OmniOS-as-a-guest as well.
> |
> | > Thanks,
> | > DAn
> |
> `-------------------
>
> Do you find the time to integrate open-vm-tools in the next stable
> version?
>
> Thanks.
>
> On Februar, 19 2015, 03:36 <Dan McDonald> wrote in [1]:
>
> > There will be only 1-3 more updates prior to the next OmniOS stable
> > (and for this time, this stable is also Long-Term Support) release.
> > If you notice anything weird about the bloody bits, please let me
> > know ASAP.  I've heard little/no complaints about the updates to
> > pkg(5), so either you're very happy, or not using it.  :)
>
> > This update will only be reaching the repo.
>
> > * omnios-build master branch, revision f3d6d48
>
> > * Git to 2.3.0.
>
> > * UnZIP fixes.
>
> > * OpenJDK7 up to update 76, build 31.
>
> > * Microtasking libraries (/lib/libmtsk*) are back as a distinct package
> >   (system/library/mtsk) now that they are not part of the (now
> open-source)
> >   Math libraries.
>
> > * illumos-omnios master branch, revision cbf73e4 (last illumos-gate
> merge 336069c)
>
> > * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.
>
> --
> Best Regards
> Alexander
> Februar, 20 2015
> ........
> [1] mid:F70C482D-68E8-43A2-BF73-86ADE4B63496 at omniti.com
> ........
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/1278f7b7/attachment.html>

From danmcd at omniti.com  Fri Feb 20 15:27:56 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 10:27:56 -0500
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
	<CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>
Message-ID: <A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>

Are you two volunteering for testing?  If so, I can see what I can do about pushing the package (without it being in an incorporation) into the bloody repo server -- assuming I can build it properly.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Feb 20, 2015, at 9:20 AM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> I will second that request as my OmniOS experiments always start as VMware VMs.
> 
> -Chip
> 
>> On Fri, Feb 20, 2015 at 3:17 AM, Alexander Lesle <groups at tierarzt-mueller.de> wrote:
>> Hello Dan McDonald and List,
>> 
>> do your remember my feature request?
>> ,-----[  ]-----
>> |
>> | at Friday, 3. Okt. 2014 17:22 <Dan McDonald>
>> | at mid:A762C059-F687-42DE-A1F7-6CF2A7F5863B at omniti.com wrote:
>> |
>> | > On Oct 3, 2014, at 9:29 AM, Alexander Lesle
>> | > <groups at tierarzt-mueller.de> wrote:
>> |
>> | >> Hello Dan McDonald and List,
>> | >>
>> | >> in my mind this was a great proposal what you done at Sep, 02.
>> | >> That's the way to build a great OmniOS near to the users.
>> | >>
>> | >> Here my request for the next release:
>> | >> It would be convenient if open-vm-tools were installed in the new
>> | >> release. http://sourceforge.net/projects/open-vm-tools/files/
>> |
>> | > Actually, this work has been done.  I dropped it on the ground:
>> |
>> | >         https://github.com/omniti-labs/omnios-build/pull/39
>> |
>> | > I will be merging this pull requests now in the master branch,
>> | > and seeing how OOod chews on it.
>> |
>> |
>> | > I know there's other work to be done for OmniOS-as-a-guest as well.
>> |
>> | > Thanks,
>> | > DAn
>> |
>> `-------------------
>> 
>> Do you find the time to integrate open-vm-tools in the next stable
>> version?
>> 
>> Thanks.
>> 
>> On Februar, 19 2015, 03:36 <Dan McDonald> wrote in [1]:
>> 
>> > There will be only 1-3 more updates prior to the next OmniOS stable
>> > (and for this time, this stable is also Long-Term Support) release.
>> > If you notice anything weird about the bloody bits, please let me
>> > know ASAP.  I've heard little/no complaints about the updates to
>> > pkg(5), so either you're very happy, or not using it.  :)
>> 
>> > This update will only be reaching the repo.
>> 
>> > * omnios-build master branch, revision f3d6d48
>> 
>> > * Git to 2.3.0.
>> 
>> > * UnZIP fixes.
>> 
>> > * OpenJDK7 up to update 76, build 31.
>> 
>> > * Microtasking libraries (/lib/libmtsk*) are back as a distinct package
>> >   (system/library/mtsk) now that they are not part of the (now open-source)
>> >   Math libraries.
>> 
>> > * illumos-omnios master branch, revision cbf73e4 (last illumos-gate merge 336069c)
>> 
>> > * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.
>> 
>> --
>> Best Regards
>> Alexander
>> Februar, 20 2015
>> ........
>> [1] mid:F70C482D-68E8-43A2-BF73-86ADE4B63496 at omniti.com
>> ........
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/579b3df3/attachment-0001.html>

From chip at innovates.com  Fri Feb 20 15:50:43 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Fri, 20 Feb 2015 09:50:43 -0600
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
	<CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>
	<A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>
Message-ID: <CALeZrrQk-xk94td5hH=zTw1qyCPOEpUTm1D59JR=QCieibSvpg@mail.gmail.com>

On Fri, Feb 20, 2015 at 9:27 AM, Dan McDonald <danmcd at omniti.com> wrote:

> Are you two volunteering for testing?  If so, I can see what I can do
> about pushing the package (without it being in an incorporation) into the
> bloody repo server -- assuming I can build it properly.
>

Absolutely.   I have several OmniOS VMs already.   At least 1/2 of them
need VMware tools.

-Chip

>
> Dan
>
> Sent from my iPhone (typos, autocorrect, and all)
>
> On Feb 20, 2015, at 9:20 AM, Schweiss, Chip <chip at innovates.com> wrote:
>
> I will second that request as my OmniOS experiments always start as VMware
> VMs.
>
> -Chip
>
> On Fri, Feb 20, 2015 at 3:17 AM, Alexander Lesle <
> groups at tierarzt-mueller.de> wrote:
>
>> Hello Dan McDonald and List,
>>
>> do your remember my feature request?
>> ,-----[  ]-----
>> |
>> | at Friday, 3. Okt. 2014 17:22 <Dan McDonald>
>> | at mid:A762C059-F687-42DE-A1F7-6CF2A7F5863B at omniti.com wrote:
>> |
>> | > On Oct 3, 2014, at 9:29 AM, Alexander Lesle
>> | > <groups at tierarzt-mueller.de> wrote:
>> |
>> | >> Hello Dan McDonald and List,
>> | >>
>> | >> in my mind this was a great proposal what you done at Sep, 02.
>> | >> That's the way to build a great OmniOS near to the users.
>> | >>
>> | >> Here my request for the next release:
>> | >> It would be convenient if open-vm-tools were installed in the new
>> | >> release. http://sourceforge.net/projects/open-vm-tools/files/
>> |
>> | > Actually, this work has been done.  I dropped it on the ground:
>> |
>> | >         https://github.com/omniti-labs/omnios-build/pull/39
>> |
>> | > I will be merging this pull requests now in the master branch,
>> | > and seeing how OOod chews on it.
>> |
>> |
>> | > I know there's other work to be done for OmniOS-as-a-guest as well.
>> |
>> | > Thanks,
>> | > DAn
>> |
>> `-------------------
>>
>> Do you find the time to integrate open-vm-tools in the next stable
>> version?
>>
>> Thanks.
>>
>> On Februar, 19 2015, 03:36 <Dan McDonald> wrote in [1]:
>>
>> > There will be only 1-3 more updates prior to the next OmniOS stable
>> > (and for this time, this stable is also Long-Term Support) release.
>> > If you notice anything weird about the bloody bits, please let me
>> > know ASAP.  I've heard little/no complaints about the updates to
>> > pkg(5), so either you're very happy, or not using it.  :)
>>
>> > This update will only be reaching the repo.
>>
>> > * omnios-build master branch, revision f3d6d48
>>
>> > * Git to 2.3.0.
>>
>> > * UnZIP fixes.
>>
>> > * OpenJDK7 up to update 76, build 31.
>>
>> > * Microtasking libraries (/lib/libmtsk*) are back as a distinct package
>> >   (system/library/mtsk) now that they are not part of the (now
>> open-source)
>> >   Math libraries.
>>
>> > * illumos-omnios master branch, revision cbf73e4 (last illumos-gate
>> merge 336069c)
>>
>> > * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.
>>
>> --
>> Best Regards
>> Alexander
>> Februar, 20 2015
>> ........
>> [1] mid:F70C482D-68E8-43A2-BF73-86ADE4B63496 at omniti.com
>> ........
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/f811f1eb/attachment.html>

From danmcd at omniti.com  Fri Feb 20 16:10:13 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 11:10:13 -0500
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <521254406.20150220101749@tierarzt-mueller.de>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
Message-ID: <50173EA8-63C6-4E47-B9AD-DDD922CFA290@omniti.com>


> On Feb 20, 2015, at 4:17 AM, Alexander Lesle <groups at tierarzt-mueller.de> wrote:
> 
> | > Actually, this work has been done.  I dropped it on the ground:
> | 
> | >         https://github.com/omniti-labs/omnios-build/pull/39
> | 
> | > I will be merging this pull requests now in the master branch,
> | > and seeing how OOod chews on it.
> | 

Ahhh... turns out, this is *IN* bloody already.  If you're running bloody, you can install "open-vm-tools".

Dan


From danmcd at omniti.com  Fri Feb 20 16:16:17 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 11:16:17 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
	OmniOS r151010
Message-ID: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>

Sometime during Q2CY2015 (no earlier than April 1st), OmniOS r151014 will be released.  At that time, support for r151010 ("previous stable") will be discontinued.  This is in accordance with the OmniOS release cycle:

	http://omnios.omniti.com/wiki.php/ReleaseCycle

If you can, please upgrade to r151012.  We *should* be able to allow a jump from r151010 to r151014, as we must support a jump from r151012 to 014 AND from 006 to 014 (because r151014 will also be the next Long-Term Support release).

As I mentioned the last time, beadm(1M) is your friend.  Creating backup BEs in case of mistakes is easy.

Thank you,
Dan McDonald -- OmniOS Engineering

From jesus at omniti.com  Fri Feb 20 16:32:42 2015
From: jesus at omniti.com (Theo Schlossnagle)
Date: Fri, 20 Feb 2015 11:32:42 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
 OmniOS r151010
In-Reply-To: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
Message-ID: <CACLsApv0gpZYsMco538GvuUYq+HCiadQidhS+ZUuwq6iaEgmwA@mail.gmail.com>

You know what would be nice?  an omnios.omniti.com/releases web page that
has the release name, date and EOL

[r151006 : LTS]
[birth <date>]
[end-of-life <date>]

[r151010 : stable]
[birth <date>]
[end-of-life 2015-04-01]

[r151013 : bloody]
[birth <date>]
[not supported]

[r151014 : LTS]
[birth "we're expecting"]
[end-of-life <date>]

We could make that nice an pretty and I think it would be very useful.

On Fri, Feb 20, 2015 at 11:16 AM, Dan McDonald <danmcd at omniti.com> wrote:

> Sometime during Q2CY2015 (no earlier than April 1st), OmniOS r151014 will
> be released.  At that time, support for r151010 ("previous stable") will be
> discontinued.  This is in accordance with the OmniOS release cycle:
>
>         http://omnios.omniti.com/wiki.php/ReleaseCycle
>
> If you can, please upgrade to r151012.  We *should* be able to allow a
> jump from r151010 to r151014, as we must support a jump from r151012 to 014
> AND from 006 to 014 (because r151014 will also be the next Long-Term
> Support release).
>
> As I mentioned the last time, beadm(1M) is your friend.  Creating backup
> BEs in case of mistakes is easy.
>
> Thank you,
> Dan McDonald -- OmniOS Engineering
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>



-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/10c5e1c0/attachment.html>

From tobi at oetiker.ch  Fri Feb 20 19:58:55 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Fri, 20 Feb 2015 20:58:55 +0100 (CET)
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
 OmniOS r151010
In-Reply-To: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
Message-ID: <alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>

Hi Dan,

Today Dan McDonald wrote:

> Sometime during Q2CY2015 (no earlier than April 1st), OmniOS
> r151014 will be released.  At that time, support for r151010
> ("previous stable") will be discontinued.  This is in accordance
> with the OmniOS release cycle:
>
> 	http://omnios.omniti.com/wiki.php/ReleaseCycle
>
> If you can, please upgrade to r151012.  We *should* be able to
> allow a jump from r151010 to r151014, as we must support a jump
> from r151012 to 014 AND from 006 to 014 (because r151014 will
> also be the next Long-Term Support release).
>
> As I mentioned the last time, beadm(1M) is your friend.  Creating
> backup BEs in case of mistakes is easy.

I do not have a spare machine to test this, but if the dramatic
network io performance regression for kvm present in r151012 has
not been fixed in r151014 then we will be stranded on r151010.

Judging from the echo on the ML, not many people seem to see this
problem, or care about it.

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From danmcd at omniti.com  Fri Feb 20 20:17:16 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 15:17:16 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
	OmniOS r151010
In-Reply-To: <alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
Message-ID: <E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>


> On Feb 20, 2015, at 2:58 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> 
> Hi Dan,
> 
> I do not have a spare machine to test this, but if the dramatic
> network io performance regression for kvm present in r151012 has
> not been fixed in r151014 then we will be stranded on r151010.

And the upstreaming of VND is likely not to happen with this release.  That blocks us from properly catching up with illumos-kvm-cmd.

There are several commits (Starting with the March 19th one we can't bring in w/o VND) here:

	https://github.com/joyent/illumos-kvm-cmd/commits/master

Currently, we only backport the most recent fix about QEMU using preadv/pwritev recklessly.  IF (big if) there are other commits we can backport, you could try it using the patches/ directory of omnios-build.  I'm thinking maybe HVM-807's may help you (provided that's indepdent of VND support).

Have you seen anything with dtrace or lockstat that could give a clue to what's holding things up?  I don't recall seeing anything observed.

> Judging from the echo on the ML, not many people seem to see this
> problem, or care about it.

I'm sorry, but that does appear to be the case.

Next week I disappear for other-customer work, but starting in March, I'm doing nothing but r151014.

Dan


From danmcd at omniti.com  Fri Feb 20 20:18:51 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 15:18:51 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
	OmniOS r151010
In-Reply-To: <alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
Message-ID: <550A9F98-ACDE-4162-B323-F72056045ADA@omniti.com>

Unicast...

Found this:

	http://echelog.com/logs/browse/smartos/1408053600

With this to say:

<rmustacc> So, I can give you a copy of the analysis.
<rmustacc> Basically the number of iovecs that we can receive is up to the virtio ring buffer size.
<rmustacc> Which is a lot more than the number of frame io vectors.
<rmustacc> And while this shouldn't generally happen, Windows has been guilt of it for non-jumbo frames.
<rmustacc> Why you would break up a <1500 byte packet into 32 iovectors is beyond me.
<nahamu> and when it does that QEMU crashes on the assert with the old code, correct?

Dan


From groups at tierarzt-mueller.de  Fri Feb 20 20:43:54 2015
From: groups at tierarzt-mueller.de (Alexander Lesle)
Date: Fri, 20 Feb 2015 21:43:54 +0100
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
	<CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>
	<A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>
Message-ID: <866483450.20150220214354@tierarzt-mueller.de>

Hello Dan McDonald and List,

thanks for your offer, but I cant do that because I am not a omnios
expert enough and my english is not very well.

On Februar, 20 2015, 16:27 <Dan McDonald> wrote in [1]:

> Are you two volunteering for testing?  If so, I can see what I can
> do about pushing the package (without it being in an incorporation)
> into the bloody repo server -- assuming I can build it properly.


> Dan

> Sent from my iPhone (typos, autocorrect, and all)

> On Feb 20, 2015, at 9:20 AM, Schweiss, Chip <chip at innovates.com> wrote:


> I will second that request as my OmniOS experiments always start as VMware VMs.


> -Chip


> On Fri, Feb 20, 2015 at 3:17 AM, Alexander Lesle
> <groups at tierarzt-mueller.de> wrote:

> Hello Dan McDonald and List,
>  
>  do your remember my feature request?
>  ,-----[  ]-----
>  |
>  | at Friday, 3. Okt. 2014 17:22 <Dan McDonald>
>  | at mid:A762C059-F687-42DE-A1F7-6CF2A7F5863B at omniti.com wrote:
>  |
>  | > On Oct 3, 2014, at 9:29 AM, Alexander Lesle
>  | > <groups at tierarzt-mueller.de> wrote:
>  |
>  | >> Hello Dan McDonald and List,
>  | >>
>  | >> in my mind this was a great proposal what you done at Sep, 02.
>  | >> That's the way to build a great OmniOS near to the users.
>  | >>
>  | >> Here my request for the next release:
>  | >> It would be convenient if open-vm-tools were installed in the new
>  | >> release. http://sourceforge.net/projects/open-vm-tools/files/
>  |
>  | > Actually, this work has been done.  I dropped it on the ground:
>  |
>  | >         https://github.com/omniti-labs/omnios-build/pull/39
>  |
>  | > I will be merging this pull requests now in the master branch,
>  | > and seeing how OOod chews on it.
>  |
>  |
>  | > I know there's other work to be done for OmniOS-as-a-guest as well.
>  |
>  | > Thanks,
>  | > DAn
>  |
>  `-------------------
>  
>  Do you find the time to integrate open-vm-tools in the next stable
>  version?
>  
>  Thanks.
>  
>  On Februar, 19 2015, 03:36 <Dan McDonald> wrote in [1]:
>  
 >> There will be only 1-3 more updates prior to the next OmniOS stable
 >> (and for this time, this stable is also Long-Term Support) release.
 >> If you notice anything weird about the bloody bits, please let me
 >> know ASAP.  I've heard little/no complaints about the updates to
 >> pkg(5), so either you're very happy, or not using it.  :)
>  
 >> This update will only be reaching the repo.
>  
 >> * omnios-build master branch, revision f3d6d48
>  
 >> * Git to 2.3.0.
>  
 >> * UnZIP fixes.
>  
 >> * OpenJDK7 up to update 76, build 31.
>  
 >> * Microtasking libraries (/lib/libmtsk*) are back as a distinct package
 >>   (system/library/mtsk) now that they are not part of the (now open-source)
 >>   Math libraries.
>  
 >> * illumos-omnios master branch, revision cbf73e4 (last illumos-gate merge 336069c)
>  
 >> * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages.
>  
>  --
>  Best Regards
>  Alexander
>  Februar, 20 2015
>  ........
>  [1] mid:F70C482D-68E8-43A2-BF73-86ADE4B63496 at omniti.com
>  ........
>  

>  _______________________________________________
>  OmniOS-discuss mailing list
>  OmniOS-discuss at lists.omniti.com
>  http://lists.omniti.com/mailman/listinfo/omnios-discuss
>  

>  

> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

>  
-- 
Best Regards
Alexander
Februar, 20 2015
........
[1] mid:A19863F3-E121-445E-BF4B-DA5C97EF3772 at omniti.com
........


From danmcd at omniti.com  Fri Feb 20 20:47:42 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 15:47:42 -0500
Subject: [OmniOS-discuss] OmniOS Bloody update for Feb 18
In-Reply-To: <866483450.20150220214354@tierarzt-mueller.de>
References: <F70C482D-68E8-43A2-BF73-86ADE4B63496@omniti.com>
	<521254406.20150220101749@tierarzt-mueller.de>
	<CALeZrrQS6qd7Wr8A8i4TLGQoV4_g1=eMmNvE7Hq_jqWq9Astjg@mail.gmail.com>
	<A19863F3-E121-445E-BF4B-DA5C97EF3772@omniti.com>
	<866483450.20150220214354@tierarzt-mueller.de>
Message-ID: <DC78B111-2074-4A63-B0DA-F1449743E5DD@omniti.com>


> On Feb 20, 2015, at 3:43 PM, Alexander Lesle <groups at tierarzt-mueller.de> wrote:
> 
> Hello Dan McDonald and List,
> 
> thanks for your offer, but I cant do that because I am not a omnios
> expert enough and my english is not very well.

If you are running the bloody version, all you need is this:

	pkg install open-vm-tools

That's it!  If you're waiting for something supported, you'll need to wait for r151014 to ship.

Dan


From hasslerd at gmx.li  Fri Feb 20 22:14:48 2015
From: hasslerd at gmx.li (Dominik Hassler)
Date: Fri, 20 Feb 2015 23:14:48 +0100
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
 OmniOS r151010
In-Reply-To: <E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
	<E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>
Message-ID: <54E7B1D8.3070009@gmx.li>

Dan,

I had a look at HVM-807 you mentioned.

Shouldn't line 251 in 'net/vnic.c' be
for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX); i++, iov++) {

instead of
for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX - 1); i++, iov++) {

As otherwise the last element of the frameio will not be used at all if
iovcnt is greater than the frameio buffer size?

Kind regards
Dominik

On 02/20/2015 09:17 PM, Dan McDonald wrote:
> 
>> On Feb 20, 2015, at 2:58 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>>
>> Hi Dan,
>>
>> I do not have a spare machine to test this, but if the dramatic
>> network io performance regression for kvm present in r151012 has
>> not been fixed in r151014 then we will be stranded on r151010.
> 
> And the upstreaming of VND is likely not to happen with this release.  That blocks us from properly catching up with illumos-kvm-cmd.
> 
> There are several commits (Starting with the March 19th one we can't bring in w/o VND) here:
> 
> 	https://github.com/joyent/illumos-kvm-cmd/commits/master
> 
> Currently, we only backport the most recent fix about QEMU using preadv/pwritev recklessly.  IF (big if) there are other commits we can backport, you could try it using the patches/ directory of omnios-build.  I'm thinking maybe HVM-807's may help you (provided that's indepdent of VND support).
> 
> Have you seen anything with dtrace or lockstat that could give a clue to what's holding things up?  I don't recall seeing anything observed.
> 
>> Judging from the echo on the ML, not many people seem to see this
>> problem, or care about it.
> 
> I'm sorry, but that does appear to be the case.
> 
> Next week I disappear for other-customer work, but starting in March, I'm doing nothing but r151014.
> 
> Dan
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xF9ECC6A5.asc
Type: application/pgp-keys
Size: 2686 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/698e4f60/attachment.bin>

From hasslerd at gmx.li  Fri Feb 20 22:33:32 2015
From: hasslerd at gmx.li (Dominik Hassler)
Date: Fri, 20 Feb 2015 23:33:32 +0100
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
 OmniOS r151010
In-Reply-To: <54E7B1D8.3070009@gmx.li>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>	<E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>
	<54E7B1D8.3070009@gmx.li>
Message-ID: <54E7B63C.2040607@gmx.li>

never mind, it's getting filled later on.

should have had a look at the whole of it before starting to bark :/

sorry for the noise!

On 02/20/2015 11:14 PM, Dominik Hassler wrote:
> Dan,
> 
> I had a look at HVM-807 you mentioned.
> 
> Shouldn't line 251 in 'net/vnic.c' be
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX); i++, iov++) {
> 
> instead of
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX - 1); i++, iov++) {
> 
> As otherwise the last element of the frameio will not be used at all if
> iovcnt is greater than the frameio buffer size?
> 
> Kind regards
> Dominik
> 
> On 02/20/2015 09:17 PM, Dan McDonald wrote:
>>
>>> On Feb 20, 2015, at 2:58 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>>>
>>> Hi Dan,
>>>
>>> I do not have a spare machine to test this, but if the dramatic
>>> network io performance regression for kvm present in r151012 has
>>> not been fixed in r151014 then we will be stranded on r151010.
>>
>> And the upstreaming of VND is likely not to happen with this release.  That blocks us from properly catching up with illumos-kvm-cmd.
>>
>> There are several commits (Starting with the March 19th one we can't bring in w/o VND) here:
>>
>> 	https://github.com/joyent/illumos-kvm-cmd/commits/master
>>
>> Currently, we only backport the most recent fix about QEMU using preadv/pwritev recklessly.  IF (big if) there are other commits we can backport, you could try it using the patches/ directory of omnios-build.  I'm thinking maybe HVM-807's may help you (provided that's indepdent of VND support).
>>
>> Have you seen anything with dtrace or lockstat that could give a clue to what's holding things up?  I don't recall seeing anything observed.
>>
>>> Judging from the echo on the ML, not many people seem to see this
>>> problem, or care about it.
>>
>> I'm sorry, but that does appear to be the case.
>>
>> Next week I disappear for other-customer work, but starting in March, I'm doing nothing but r151014.
>>
>> Dan
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xF9ECC6A5.asc
Type: application/pgp-keys
Size: 2686 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/8c8b2782/attachment.bin>

From wverb73 at gmail.com  Sat Feb 21 01:48:29 2015
From: wverb73 at gmail.com (W Verb)
Date: Fri, 20 Feb 2015 17:48:29 -0800
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
Message-ID: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>

Hello all,

Each of the things in the subject line are:
1: Horrendously broken
2: Have an extremely poor short-term outlook
3: Will take a huge investment of time by intelligent, dedicated, and
insightful people to fix.

It's common knowledge that the ixgbe driver in omnios/illumos/opensolaris
is broke as hell. The point of this message is not to complain. The point
is to pass on a configuration that is working for me, albeit in a
screwed-up degraded fashion.

I have four ESXi 5.5u2 host servers with 1 Intel PCI-e quad-gigabit NIC
installed in each. Three of the gigabit ports on each client are dedicated
to carry iSCSI traffic between each host and a single storage server.

The storage server is based on a Supermicro X10SLM-F mainboard, which has
three PCI-e slots. Two of the slots are used for storage controllers, and a
single slot is used for an Intel X520 dual-port fiber 10G NIC.

Previously, I had a single storage controller and two quad-gig NICs
installed in the storage server, and was able to get close to line-rate on
multipath iSCSI with three host clients. But when I added the fourth, I
upgraded to 10G.

After installation and configuration, I observed all kinds of bad behavior
in the network traffic between the hosts and the server. All of this bad
behavior is traced to the ixgbe driver on the storage server. Without going
into the full troubleshooting process, here are my takeaways:

1: The only tuning factor that appears to have significant effect on the
driver is MTU size. This applies to both the MTU of the ixgbe NIC as well
as the MTU of the 1-gig NICs used in the hosts.

2: I have seen best performance with the MTU on the ixgbe set to 1000 bytes
(yes, 1k). The MTU on the ESXi interfaces is set to 3000 bytes.

3: Setting 9000 byte MTUs on both sides results in about 150MB/s write
speeds on a a linux vmware guest running a 10GB dd operation. But read
speeds are at 5MB/s.

4: Testing of dd operations on the storage server itself shows that the
filesystem is capable of performing 500MB/s+ reads and writes.

5: After setting the MTUs listed in point 2, I am able to get 270-300MB/s
writes on the guest OS, and ~200MB/s reads. Not perfect, but I'll take it.

6: No /etc/system or other kernel tunings are in use.

7: Delayed ACK, Nagle, and L2 flow control tests had no effect.

8: pkg -u was performed before all tests, so I should be using the latest
kernel code, etc.

9: When capturing traffic on omnios, I used the CSW distribution of
tcpdump. It's worth noting that unlike EVERY ... OTHER ... IMPLEMENTATION
... of tcpdump I've ever used (BSD flavors, OSX, various linux distros,
various embedded distros), libpcap doesn't appear to get individual frame
reports from the omnios kernel, and so aggregates multi-frame TCP segments
into a single record. This has the appearance of 20-60kB frames being
transmitted by omnios when reading a packet capture with Wireshark. I
cannot tell you how irritating this is when troubleshooting network issues.

10: At the wire level, the speed problems are clearly due to pauses in
response time by omnios. At 9000 byte frame sizes, I see a good number of
duplicate ACKs and fast retransmits during read operations (when omnios is
transmitting). But below about a 4100-byte MTU on omnios (which seems to
correlate to 4096-byte iSCSI block transfers), the transmission errors fade
away and we only see the transmission pause problem.

I'm in the process of aggregating the 10G ports and performing some IO
testing with the vmware IO performance tool. That should show the
performance of the 10G NIC when both physical ports are in use, and
hopefully get me some more granularity on the MTU settings.

If anyone has a list of kernel tuning parameters to test, I'm happy to try
them out and report back. I've found a variety of suggestions online, but
between Illumos, solaris, openindiana, Nexenta, opensolaris, etc, the
supported variables are, um, inconsistent.

-Warren V
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/bc7dda80/attachment-0001.html>

From danmcd at omniti.com  Sat Feb 21 02:10:18 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 21:10:18 -0500
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
Message-ID: <B2C77D35-6134-4335-BCF2-F81671748F8F@omniti.com>

You should PLEASE share your original note with the illumos developer's list.

Also, please keep in mind that it is VERY possible you're seeing crosscall effects where the interrupt-servicing CPU core is not on the same PCIe bus as where you're card is plugged in.

I never got a chance to perform these tests fully when I *had* ixgbe HW handy, but I observed bizarre improvements, or the disappearance of bizarre effects, if I:

	- disabled the HT-inspired CPUs (psradm -f <HT CPUs>)

	- Disabled one of the two CPUs (again, using psradm).

You may wish to try messing around with what OS-reported CPUs are on your Romley (Xeon E5) system.

I will also note that it's high time for illumos to pull in the ixgbe updates from upstream.  Intel is NOT being very helpful here, partially because of fear-of-Oracle, and partially because there aren't enough illumos customers to make a dent in their HW sales.

In the past, illumos developers have found the time to yank in the newest driver updates from upstream.  That hasn't happened recently. For the record, OmniTI might be able to contribute here IF AND ONLY IF a sufficiently paying customer motivates us. I suspect the same answer (sufficiently paying customer) applies to engineers from any other illumos shop as well.

Dan


From danmcd at omniti.com  Sat Feb 21 02:12:03 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 20 Feb 2015 21:12:03 -0500
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <B2C77D35-6134-4335-BCF2-F81671748F8F@omniti.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<B2C77D35-6134-4335-BCF2-F81671748F8F@omniti.com>
Message-ID: <2A2FE1C0-405F-4B95-A2E6-1257FC9BD075@omniti.com>


> On Feb 20, 2015, at 9:10 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> I never got a chance to perform these tests fully when I *had* ixgbe HW handy, but I observed bizarre improvements, or the disappearance of bizarre effects, if I:
> 
> 	- disabled the HT-inspired CPUs (psradm -f <HT CPUs>)
> 
> 	- Disabled one of the two CPUs (again, using psradm).
> 
> You may wish to try messing around with what OS-reported CPUs are on your Romley (Xeon E5) system.

To see the layouts, the psrinfo(1M) command is your friend, especially "psrinfo -vp".

Dan


From cks at cs.toronto.edu  Sat Feb 21 03:25:59 2015
From: cks at cs.toronto.edu (Chris Siebenmann)
Date: Fri, 20 Feb 2015 22:25:59 -0500
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: Your message of Fri, 20 Feb 2015 17:48:29 -0800.
	<CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
Message-ID: <20150221032559.727D07A0792@apps0.cs.toronto.edu>

> After installation and configuration, I observed all kinds of bad behavior
> in the network traffic between the hosts and the server. All of this bad
> behavior is traced to the ixgbe driver on the storage server. Without going
> into the full troubleshooting process, here are my takeaways:
[...]

 For what it's worth, we managed to achieve much better line rates on
copper 10G ixgbe hardware of various descriptions between OmniOS
and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
believe OmniOS could do TCP at full line rate but I think we managed 700+
Mbytes/sec on both transmit and receive and we got basically disk-limited
speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
OmniOS iSCSI initiator, Linux iSCSI targets).

 I don't believe we did any specific kernel tuning (and in fact some of
our attempts to fiddle ixgbe driver parameters blew up in our face).
We did tune iSCSI connection parameters to increase various buffer
sizes so that ZFS could do even large single operations in single iSCSI
transactions. (More details available if people are interested.)

> 10: At the wire level, the speed problems are clearly due to pauses in
> response time by omnios. At 9000 byte frame sizes, I see a good number
> of duplicate ACKs and fast retransmits during read operations (when
> omnios is transmitting). But below about a 4100-byte MTU on omnios
> (which seems to correlate to 4096-byte iSCSI block transfers), the
> transmission errors fade away and we only see the transmission pause
> problem.

 This is what really attracted my attention. In our OmniOS setup, our
specific Intel hardware had ixgbe driver issues that could cause
activity stalls during once-a-second link heartbeat checks. This
obviously had an effect at the TCP and iSCSI layers. My initial message
to illumos-developer sparked a potentially interesting discussion:

http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/

If you think this is a possibility in your setup, I've put the DTrace
script I used to hunt for this up on the web:

	http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d

This isn't the only potential source of driver stalls by any means, it's
just the one I found. You may also want to look at lockstat in general,
as information it reported is what led us to look specifically at the
ixgbe code here.

(If you suspect kernel/driver issues, lockstat combined with kernel
source is a really excellent resource.)

	- cks

From wverb73 at gmail.com  Sat Feb 21 05:46:51 2015
From: wverb73 at gmail.com (W Verb)
Date: Fri, 20 Feb 2015 21:46:51 -0800
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <20150221032559.727D07A0792@apps0.cs.toronto.edu>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<20150221032559.727D07A0792@apps0.cs.toronto.edu>
Message-ID: <CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>

Hello All,

Thank you for your replies.
I tried a few things, and found the following:

1: Disabling hyperthreading support in the BIOS drops performance overall
by a factor of 4.
2: Disabling VT support also seems to have some effect, although it appears
to be minor. But this has the amusing side effect of fixing the hangs I've
been experiencing with fast reboot. Probably by disabling kvm.
3: The performance tests are a bit tricky to quantify because of caching
effects. In fact, I'm not entirely sure what is happening here. It's just
best to describe what I'm seeing:

The commands I'm using to test are
dd if=/dev/zero of=./test.dd bs=2M count=5000
dd of=/dev/null if=./test.dd bs=2M count=5000
The host vm is running Centos 6.6, and has the latest vmtools installed.
There is a host cache on an SSD local to the host that is also in place.
Disabling the host cache didn't immediately have an effect as far as I
could see.

The host MTU set to 3000 on all iSCSI interfaces for all tests.

Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
yields an average speed over three tests of 137MB/s. The read test yields
an average over three tests of 5MB/s.

Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
140MB/s, and the read tests yield 53MB/s. It's important to note here that
if I cut the read test short at only 2-3GB, I get results upwards of
350MB/s, which I assume is local cache-related distortion.

Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield about
142MB/s.
Test 4: MTU of 1000: Read test at 182MB/s.
Test 5: MTU of 900: Read test at 130 MB/s.
Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now consistently
at about 300MB/s.
Test 7: MTU of 1200: Read test at 124MB/s.
Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.

A few final notes:
L1ARC grabs about 10GB of RAM during the tests, so there's definitely some
read caching going on.
The write operations are easier to observe with iostat, and I'm seeing io
rates that closely correlate with the network write speeds.


Chris, thanks for your specific details. I'd appreciate it if you could
tell me which copper NIC you tried, as well as to pass on the iSCSI tuning
parameters.

I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead of
the 82599 in the X520. If I get similar results with my fiber transcievers,
I'll see if I can get a hold of copper ones.

But I should mention that I did indeed look at PHY/MAC error rates, and
they are nil.

-Warren V

On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
wrote:

> > After installation and configuration, I observed all kinds of bad
> behavior
> > in the network traffic between the hosts and the server. All of this bad
> > behavior is traced to the ixgbe driver on the storage server. Without
> going
> > into the full troubleshooting process, here are my takeaways:
> [...]
>
>  For what it's worth, we managed to achieve much better line rates on
> copper 10G ixgbe hardware of various descriptions between OmniOS
> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
> believe OmniOS could do TCP at full line rate but I think we managed 700+
> Mbytes/sec on both transmit and receive and we got basically disk-limited
> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
> OmniOS iSCSI initiator, Linux iSCSI targets).
>
>  I don't believe we did any specific kernel tuning (and in fact some of
> our attempts to fiddle ixgbe driver parameters blew up in our face).
> We did tune iSCSI connection parameters to increase various buffer
> sizes so that ZFS could do even large single operations in single iSCSI
> transactions. (More details available if people are interested.)
>
> > 10: At the wire level, the speed problems are clearly due to pauses in
> > response time by omnios. At 9000 byte frame sizes, I see a good number
> > of duplicate ACKs and fast retransmits during read operations (when
> > omnios is transmitting). But below about a 4100-byte MTU on omnios
> > (which seems to correlate to 4096-byte iSCSI block transfers), the
> > transmission errors fade away and we only see the transmission pause
> > problem.
>
>  This is what really attracted my attention. In our OmniOS setup, our
> specific Intel hardware had ixgbe driver issues that could cause
> activity stalls during once-a-second link heartbeat checks. This
> obviously had an effect at the TCP and iSCSI layers. My initial message
> to illumos-developer sparked a potentially interesting discussion:
>
>
> http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/
>
> If you think this is a possibility in your setup, I've put the DTrace
> script I used to hunt for this up on the web:
>
>         http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>
> This isn't the only potential source of driver stalls by any means, it's
> just the one I found. You may also want to look at lockstat in general,
> as information it reported is what led us to look specifically at the
> ixgbe code here.
>
> (If you suspect kernel/driver issues, lockstat combined with kernel
> source is a really excellent resource.)
>
>         - cks
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150220/5876a4a9/attachment.html>

From chip at innovates.com  Sat Feb 21 15:59:57 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Sat, 21 Feb 2015 09:59:57 -0600
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<20150221032559.727D07A0792@apps0.cs.toronto.edu>
	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
Message-ID: <CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>

I can't say I totally agree with your performance assessment.   I run Intel
X520 in all my OmniOS boxes.

Here is a capture of nfssvrtop I made while running many storage vMotions
between two OmniOS boxes hosting NFS datastores.   This is a 10 host VMware
cluster.  Both OmniOS boxes are dual 10G connected with copper twin-ax to
the in rack Nexus 5010.

VMware does 100% sync writes, I use ZeusRAM SSDs for log devices.

-Chip

2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB, swrite: 15985    KB,
awrite: 1875455  KB

Ver     Client           NFSOPS   Reads SWrites AWrites Commits   Rd_bw
SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%

4       10.28.17.105          0       0       0       0       0
0       0       0       0       0       0       0       0

4       10.28.17.215          0       0       0       0       0
    0       0       0       0       0       0       0       0

4       10.28.17.213          0       0       0       0       0
0       0       0       0       0       0       0       0

4       10.28.16.151          0       0       0       0       0
    0       0       0       0       0       0       0       0

4       all                   1       0       0       0       0
0       0       0       0       0       0       0       0

3       10.28.16.175          3       0       3       0       0
    1      11       0    4806      48       0       0      85

3       10.28.16.183          6       0       6       0       0       3
162       0     549     124       0       0      73

3       10.28.16.180         11       0      10       0       0
    3      27       0     776      89       0       0      67

3       10.28.16.176         28       2      26       0       0      10
405       0    2572     198       0       0     100

3       10.28.16.178       4606    4602       4       0       0
294534       3       0     723      49       0       0      99

3       10.28.16.179       4905    4879      26       0       0  312208
311       0     735     271       0       0      99

3       10.28.16.181       5515    5502      13       0       0
352107      77       0      89      87       0       0      99

3       10.28.16.184      12095   12059      10       0       0
763014      39       0     249     147       0       0      99

3       10.28.58.1        15401    6040     116    6354      53  191605
474  202346     192      96     144      83      99

3       all               42574   33086     217    6354      53 *1913488*
1582  202300     348     138     153     105      99




On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com> wrote:

> Hello All,
>
> Thank you for your replies.
> I tried a few things, and found the following:
>
> 1: Disabling hyperthreading support in the BIOS drops performance overall
> by a factor of 4.
> 2: Disabling VT support also seems to have some effect, although it
> appears to be minor. But this has the amusing side effect of fixing the
> hangs I've been experiencing with fast reboot. Probably by disabling kvm.
> 3: The performance tests are a bit tricky to quantify because of caching
> effects. In fact, I'm not entirely sure what is happening here. It's just
> best to describe what I'm seeing:
>
> The commands I'm using to test are
> dd if=/dev/zero of=./test.dd bs=2M count=5000
> dd of=/dev/null if=./test.dd bs=2M count=5000
> The host vm is running Centos 6.6, and has the latest vmtools installed.
> There is a host cache on an SSD local to the host that is also in place.
> Disabling the host cache didn't immediately have an effect as far as I
> could see.
>
> The host MTU set to 3000 on all iSCSI interfaces for all tests.
>
> Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
> yields an average speed over three tests of 137MB/s. The read test yields
> an average over three tests of 5MB/s.
>
> Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
> 140MB/s, and the read tests yield 53MB/s. It's important to note here that
> if I cut the read test short at only 2-3GB, I get results upwards of
> 350MB/s, which I assume is local cache-related distortion.
>
> Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield
> about 142MB/s.
> Test 4: MTU of 1000: Read test at 182MB/s.
> Test 5: MTU of 900: Read test at 130 MB/s.
> Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now
> consistently at about 300MB/s.
> Test 7: MTU of 1200: Read test at 124MB/s.
> Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>
> A few final notes:
> L1ARC grabs about 10GB of RAM during the tests, so there's definitely some
> read caching going on.
> The write operations are easier to observe with iostat, and I'm seeing io
> rates that closely correlate with the network write speeds.
>
>
> Chris, thanks for your specific details. I'd appreciate it if you could
> tell me which copper NIC you tried, as well as to pass on the iSCSI tuning
> parameters.
>
> I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead of
> the 82599 in the X520. If I get similar results with my fiber transcievers,
> I'll see if I can get a hold of copper ones.
>
> But I should mention that I did indeed look at PHY/MAC error rates, and
> they are nil.
>
> -Warren V
>
> On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
> wrote:
>
>> > After installation and configuration, I observed all kinds of bad
>> behavior
>> > in the network traffic between the hosts and the server. All of this bad
>> > behavior is traced to the ixgbe driver on the storage server. Without
>> going
>> > into the full troubleshooting process, here are my takeaways:
>> [...]
>>
>>  For what it's worth, we managed to achieve much better line rates on
>> copper 10G ixgbe hardware of various descriptions between OmniOS
>> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
>> believe OmniOS could do TCP at full line rate but I think we managed 700+
>> Mbytes/sec on both transmit and receive and we got basically disk-limited
>> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
>> OmniOS iSCSI initiator, Linux iSCSI targets).
>>
>>  I don't believe we did any specific kernel tuning (and in fact some of
>> our attempts to fiddle ixgbe driver parameters blew up in our face).
>> We did tune iSCSI connection parameters to increase various buffer
>> sizes so that ZFS could do even large single operations in single iSCSI
>> transactions. (More details available if people are interested.)
>>
>> > 10: At the wire level, the speed problems are clearly due to pauses in
>> > response time by omnios. At 9000 byte frame sizes, I see a good number
>> > of duplicate ACKs and fast retransmits during read operations (when
>> > omnios is transmitting). But below about a 4100-byte MTU on omnios
>> > (which seems to correlate to 4096-byte iSCSI block transfers), the
>> > transmission errors fade away and we only see the transmission pause
>> > problem.
>>
>>  This is what really attracted my attention. In our OmniOS setup, our
>> specific Intel hardware had ixgbe driver issues that could cause
>> activity stalls during once-a-second link heartbeat checks. This
>> obviously had an effect at the TCP and iSCSI layers. My initial message
>> to illumos-developer sparked a potentially interesting discussion:
>>
>>
>> http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/
>>
>> If you think this is a possibility in your setup, I've put the DTrace
>> script I used to hunt for this up on the web:
>>
>>         http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>>
>> This isn't the only potential source of driver stalls by any means, it's
>> just the one I found. You may also want to look at lockstat in general,
>> as information it reported is what led us to look specifically at the
>> ixgbe code here.
>>
>> (If you suspect kernel/driver issues, lockstat combined with kernel
>> source is a really excellent resource.)
>>
>>         - cks
>>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150221/c6c59981/attachment-0001.html>

From tobi at oetiker.ch  Sat Feb 21 21:13:54 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Sat, 21 Feb 2015 22:13:54 +0100 (CET)
Subject: [OmniOS-discuss] recovering a faulted disk
Message-ID: <alpine.DEB.2.10.1502212207340.14595@froburg.oetiker.ch>

If zfs sees too many errors on a disk, it will 'FAULT'
the disk, replace it with a spare disk and start resilvering.

We have one system where this keeps happening ... as explained in

http://lists.omniti.com/pipermail/omnios-discuss/2014-March/002413.html

We have since replaced the firmware on the controllers, and
increassed the fan speed, to make sure the contollers are not hot.

The disk, in any event seem to be ok, since they do not exhibit any
problem when testing afterwards.

So my question is, can I somehow convince ZFS that the disk it just
FAULTED is actually OK, and it should just fixup the bits that have
changed since it abandoned the particular disk ?

The reason I am asking this is that resilvering takes quite a lot
of time with the stuff we have on these drives, a few days at least
... and just a few hours ago, the scond drive in our RAIDZ2 got
faulted ...


cheers
tobi
-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From wverb73 at gmail.com  Mon Feb 23 01:54:58 2015
From: wverb73 at gmail.com (W Verb)
Date: Sun, 22 Feb 2015 17:54:58 -0800
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<20150221032559.727D07A0792@apps0.cs.toronto.edu>
	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
	<CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>
Message-ID: <CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>

By the way, to those of you who have working setups: please send me
your pool/volume settings, interface linkprops, and any kernel tuning
parameters you may have set.

Thanks,
Warren V

On Sat, Feb 21, 2015 at 7:59 AM, Schweiss, Chip <chip at innovates.com> wrote:
> I can't say I totally agree with your performance assessment.   I run Intel
> X520 in all my OmniOS boxes.
>
> Here is a capture of nfssvrtop I made while running many storage vMotions
> between two OmniOS boxes hosting NFS datastores.   This is a 10 host VMware
> cluster.  Both OmniOS boxes are dual 10G connected with copper twin-ax to
> the in rack Nexus 5010.
>
> VMware does 100% sync writes, I use ZeusRAM SSDs for log devices.
>
> -Chip
>
> 2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB, swrite: 15985    KB,
> awrite: 1875455  KB
>
> Ver     Client           NFSOPS   Reads SWrites AWrites Commits   Rd_bw
> SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%
>
> 4       10.28.17.105          0       0       0       0       0       0
> 0       0       0       0       0       0       0
>
> 4       10.28.17.215          0       0       0       0       0       0
> 0       0       0       0       0       0       0
>
> 4       10.28.17.213          0       0       0       0       0       0
> 0       0       0       0       0       0       0
>
> 4       10.28.16.151          0       0       0       0       0       0
> 0       0       0       0       0       0       0
>
> 4       all                   1       0       0       0       0       0
> 0       0       0       0       0       0       0
>
> 3       10.28.16.175          3       0       3       0       0       1
> 11       0    4806      48       0       0      85
>
> 3       10.28.16.183          6       0       6       0       0       3
> 162       0     549     124       0       0      73
>
> 3       10.28.16.180         11       0      10       0       0       3
> 27       0     776      89       0       0      67
>
> 3       10.28.16.176         28       2      26       0       0      10
> 405       0    2572     198       0       0     100
>
> 3       10.28.16.178       4606    4602       4       0       0  294534
> 3       0     723      49       0       0      99
>
> 3       10.28.16.179       4905    4879      26       0       0  312208
> 311       0     735     271       0       0      99
>
> 3       10.28.16.181       5515    5502      13       0       0  352107
> 77       0      89      87       0       0      99
>
> 3       10.28.16.184      12095   12059      10       0       0  763014
> 39       0     249     147       0       0      99
>
> 3       10.28.58.1        15401    6040     116    6354      53  191605
> 474  202346     192      96     144      83      99
>
> 3       all               42574   33086     217    6354      53 1913488
> 1582  202300     348     138     153     105      99
>
>
>
>
>
> On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com> wrote:
>>
>> Hello All,
>>
>> Thank you for your replies.
>> I tried a few things, and found the following:
>>
>> 1: Disabling hyperthreading support in the BIOS drops performance overall
>> by a factor of 4.
>> 2: Disabling VT support also seems to have some effect, although it
>> appears to be minor. But this has the amusing side effect of fixing the
>> hangs I've been experiencing with fast reboot. Probably by disabling kvm.
>> 3: The performance tests are a bit tricky to quantify because of caching
>> effects. In fact, I'm not entirely sure what is happening here. It's just
>> best to describe what I'm seeing:
>>
>> The commands I'm using to test are
>> dd if=/dev/zero of=./test.dd bs=2M count=5000
>> dd of=/dev/null if=./test.dd bs=2M count=5000
>> The host vm is running Centos 6.6, and has the latest vmtools installed.
>> There is a host cache on an SSD local to the host that is also in place.
>> Disabling the host cache didn't immediately have an effect as far as I could
>> see.
>>
>> The host MTU set to 3000 on all iSCSI interfaces for all tests.
>>
>> Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
>> yields an average speed over three tests of 137MB/s. The read test yields an
>> average over three tests of 5MB/s.
>>
>> Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
>> 140MB/s, and the read tests yield 53MB/s. It's important to note here that
>> if I cut the read test short at only 2-3GB, I get results upwards of
>> 350MB/s, which I assume is local cache-related distortion.
>>
>> Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield
>> about 142MB/s.
>> Test 4: MTU of 1000: Read test at 182MB/s.
>> Test 5: MTU of 900: Read test at 130 MB/s.
>> Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now
>> consistently at about 300MB/s.
>> Test 7: MTU of 1200: Read test at 124MB/s.
>> Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>>
>> A few final notes:
>> L1ARC grabs about 10GB of RAM during the tests, so there's definitely some
>> read caching going on.
>> The write operations are easier to observe with iostat, and I'm seeing io
>> rates that closely correlate with the network write speeds.
>>
>>
>> Chris, thanks for your specific details. I'd appreciate it if you could
>> tell me which copper NIC you tried, as well as to pass on the iSCSI tuning
>> parameters.
>>
>> I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead of
>> the 82599 in the X520. If I get similar results with my fiber transcievers,
>> I'll see if I can get a hold of copper ones.
>>
>> But I should mention that I did indeed look at PHY/MAC error rates, and
>> they are nil.
>>
>> -Warren V
>>
>> On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
>> wrote:
>>>
>>> > After installation and configuration, I observed all kinds of bad
>>> > behavior
>>> > in the network traffic between the hosts and the server. All of this
>>> > bad
>>> > behavior is traced to the ixgbe driver on the storage server. Without
>>> > going
>>> > into the full troubleshooting process, here are my takeaways:
>>> [...]
>>>
>>>  For what it's worth, we managed to achieve much better line rates on
>>> copper 10G ixgbe hardware of various descriptions between OmniOS
>>> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
>>> believe OmniOS could do TCP at full line rate but I think we managed 700+
>>> Mbytes/sec on both transmit and receive and we got basically disk-limited
>>> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
>>> OmniOS iSCSI initiator, Linux iSCSI targets).
>>>
>>>  I don't believe we did any specific kernel tuning (and in fact some of
>>> our attempts to fiddle ixgbe driver parameters blew up in our face).
>>> We did tune iSCSI connection parameters to increase various buffer
>>> sizes so that ZFS could do even large single operations in single iSCSI
>>> transactions. (More details available if people are interested.)
>>>
>>> > 10: At the wire level, the speed problems are clearly due to pauses in
>>> > response time by omnios. At 9000 byte frame sizes, I see a good number
>>> > of duplicate ACKs and fast retransmits during read operations (when
>>> > omnios is transmitting). But below about a 4100-byte MTU on omnios
>>> > (which seems to correlate to 4096-byte iSCSI block transfers), the
>>> > transmission errors fade away and we only see the transmission pause
>>> > problem.
>>>
>>>  This is what really attracted my attention. In our OmniOS setup, our
>>> specific Intel hardware had ixgbe driver issues that could cause
>>> activity stalls during once-a-second link heartbeat checks. This
>>> obviously had an effect at the TCP and iSCSI layers. My initial message
>>> to illumos-developer sparked a potentially interesting discussion:
>>>
>>>
>>> http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/
>>>
>>> If you think this is a possibility in your setup, I've put the DTrace
>>> script I used to hunt for this up on the web:
>>>
>>>         http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>>>
>>> This isn't the only potential source of driver stalls by any means, it's
>>> just the one I found. You may also want to look at lockstat in general,
>>> as information it reported is what led us to look specifically at the
>>> ixgbe code here.
>>>
>>> (If you suspect kernel/driver issues, lockstat combined with kernel
>>> source is a really excellent resource.)
>>>
>>>         - cks
>>
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>

From nagele at wildbit.com  Mon Feb 23 04:37:46 2015
From: nagele at wildbit.com (Chris Nagele)
Date: Sun, 22 Feb 2015 23:37:46 -0500
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<20150221032559.727D07A0792@apps0.cs.toronto.edu>
	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
	<CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>
	<CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>
Message-ID: <CAHfYOdVDx=4ezxM6MymZ9tfPf=gujPACEQA=S_vGBbchgU6Dwg@mail.gmail.com>

Is the issue here only related to iSCSI? We've used the X520's for NFS
for a couple of years and it has worked really well for us.

Not sure if this is an accurate test, but iperf shows the following
results for me:

Over 1GbE:

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec    981 MBytes    823 Mbits/sec

Over 10GbE on the same machines:

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  9.42 GBytes  8.09 Gbits/sec

I could be going in the wrong direction here, but I was curious as
well since we rely on 10G heavily.

Chris


On Sun, Feb 22, 2015 at 8:54 PM, W Verb <wverb73 at gmail.com> wrote:
> By the way, to those of you who have working setups: please send me
> your pool/volume settings, interface linkprops, and any kernel tuning
> parameters you may have set.
>
> Thanks,
> Warren V
>
> On Sat, Feb 21, 2015 at 7:59 AM, Schweiss, Chip <chip at innovates.com> wrote:
>> I can't say I totally agree with your performance assessment.   I run Intel
>> X520 in all my OmniOS boxes.
>>
>> Here is a capture of nfssvrtop I made while running many storage vMotions
>> between two OmniOS boxes hosting NFS datastores.   This is a 10 host VMware
>> cluster.  Both OmniOS boxes are dual 10G connected with copper twin-ax to
>> the in rack Nexus 5010.
>>
>> VMware does 100% sync writes, I use ZeusRAM SSDs for log devices.
>>
>> -Chip
>>
>> 2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB, swrite: 15985    KB,
>> awrite: 1875455  KB
>>
>> Ver     Client           NFSOPS   Reads SWrites AWrites Commits   Rd_bw
>> SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%
>>
>> 4       10.28.17.105          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.17.215          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.17.213          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.16.151          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       all                   1       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 3       10.28.16.175          3       0       3       0       0       1
>> 11       0    4806      48       0       0      85
>>
>> 3       10.28.16.183          6       0       6       0       0       3
>> 162       0     549     124       0       0      73
>>
>> 3       10.28.16.180         11       0      10       0       0       3
>> 27       0     776      89       0       0      67
>>
>> 3       10.28.16.176         28       2      26       0       0      10
>> 405       0    2572     198       0       0     100
>>
>> 3       10.28.16.178       4606    4602       4       0       0  294534
>> 3       0     723      49       0       0      99
>>
>> 3       10.28.16.179       4905    4879      26       0       0  312208
>> 311       0     735     271       0       0      99
>>
>> 3       10.28.16.181       5515    5502      13       0       0  352107
>> 77       0      89      87       0       0      99
>>
>> 3       10.28.16.184      12095   12059      10       0       0  763014
>> 39       0     249     147       0       0      99
>>
>> 3       10.28.58.1        15401    6040     116    6354      53  191605
>> 474  202346     192      96     144      83      99
>>
>> 3       all               42574   33086     217    6354      53 1913488
>> 1582  202300     348     138     153     105      99
>>
>>
>>
>>
>>
>> On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com> wrote:
>>>
>>> Hello All,
>>>
>>> Thank you for your replies.
>>> I tried a few things, and found the following:
>>>
>>> 1: Disabling hyperthreading support in the BIOS drops performance overall
>>> by a factor of 4.
>>> 2: Disabling VT support also seems to have some effect, although it
>>> appears to be minor. But this has the amusing side effect of fixing the
>>> hangs I've been experiencing with fast reboot. Probably by disabling kvm.
>>> 3: The performance tests are a bit tricky to quantify because of caching
>>> effects. In fact, I'm not entirely sure what is happening here. It's just
>>> best to describe what I'm seeing:
>>>
>>> The commands I'm using to test are
>>> dd if=/dev/zero of=./test.dd bs=2M count=5000
>>> dd of=/dev/null if=./test.dd bs=2M count=5000
>>> The host vm is running Centos 6.6, and has the latest vmtools installed.
>>> There is a host cache on an SSD local to the host that is also in place.
>>> Disabling the host cache didn't immediately have an effect as far as I could
>>> see.
>>>
>>> The host MTU set to 3000 on all iSCSI interfaces for all tests.
>>>
>>> Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
>>> yields an average speed over three tests of 137MB/s. The read test yields an
>>> average over three tests of 5MB/s.
>>>
>>> Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
>>> 140MB/s, and the read tests yield 53MB/s. It's important to note here that
>>> if I cut the read test short at only 2-3GB, I get results upwards of
>>> 350MB/s, which I assume is local cache-related distortion.
>>>
>>> Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield
>>> about 142MB/s.
>>> Test 4: MTU of 1000: Read test at 182MB/s.
>>> Test 5: MTU of 900: Read test at 130 MB/s.
>>> Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now
>>> consistently at about 300MB/s.
>>> Test 7: MTU of 1200: Read test at 124MB/s.
>>> Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>>>
>>> A few final notes:
>>> L1ARC grabs about 10GB of RAM during the tests, so there's definitely some
>>> read caching going on.
>>> The write operations are easier to observe with iostat, and I'm seeing io
>>> rates that closely correlate with the network write speeds.
>>>
>>>
>>> Chris, thanks for your specific details. I'd appreciate it if you could
>>> tell me which copper NIC you tried, as well as to pass on the iSCSI tuning
>>> parameters.
>>>
>>> I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead of
>>> the 82599 in the X520. If I get similar results with my fiber transcievers,
>>> I'll see if I can get a hold of copper ones.
>>>
>>> But I should mention that I did indeed look at PHY/MAC error rates, and
>>> they are nil.
>>>
>>> -Warren V
>>>
>>> On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
>>> wrote:
>>>>
>>>> > After installation and configuration, I observed all kinds of bad
>>>> > behavior
>>>> > in the network traffic between the hosts and the server. All of this
>>>> > bad
>>>> > behavior is traced to the ixgbe driver on the storage server. Without
>>>> > going
>>>> > into the full troubleshooting process, here are my takeaways:
>>>> [...]
>>>>
>>>>  For what it's worth, we managed to achieve much better line rates on
>>>> copper 10G ixgbe hardware of various descriptions between OmniOS
>>>> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
>>>> believe OmniOS could do TCP at full line rate but I think we managed 700+
>>>> Mbytes/sec on both transmit and receive and we got basically disk-limited
>>>> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
>>>> OmniOS iSCSI initiator, Linux iSCSI targets).
>>>>
>>>>  I don't believe we did any specific kernel tuning (and in fact some of
>>>> our attempts to fiddle ixgbe driver parameters blew up in our face).
>>>> We did tune iSCSI connection parameters to increase various buffer
>>>> sizes so that ZFS could do even large single operations in single iSCSI
>>>> transactions. (More details available if people are interested.)
>>>>
>>>> > 10: At the wire level, the speed problems are clearly due to pauses in
>>>> > response time by omnios. At 9000 byte frame sizes, I see a good number
>>>> > of duplicate ACKs and fast retransmits during read operations (when
>>>> > omnios is transmitting). But below about a 4100-byte MTU on omnios
>>>> > (which seems to correlate to 4096-byte iSCSI block transfers), the
>>>> > transmission errors fade away and we only see the transmission pause
>>>> > problem.
>>>>
>>>>  This is what really attracted my attention. In our OmniOS setup, our
>>>> specific Intel hardware had ixgbe driver issues that could cause
>>>> activity stalls during once-a-second link heartbeat checks. This
>>>> obviously had an effect at the TCP and iSCSI layers. My initial message
>>>> to illumos-developer sparked a potentially interesting discussion:
>>>>
>>>>
>>>> http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/
>>>>
>>>> If you think this is a possibility in your setup, I've put the DTrace
>>>> script I used to hunt for this up on the web:
>>>>
>>>>         http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>>>>
>>>> This isn't the only potential source of driver stalls by any means, it's
>>>> just the one I found. You may also want to look at lockstat in general,
>>>> as information it reported is what led us to look specifically at the
>>>> ixgbe code here.
>>>>
>>>> (If you suspect kernel/driver issues, lockstat combined with kernel
>>>> source is a really excellent resource.)
>>>>
>>>>         - cks
>>>
>>>
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From jg at osn.de  Mon Feb 23 09:15:20 2015
From: jg at osn.de (Joerg Goltermann)
Date: Mon, 23 Feb 2015 10:15:20 +0100
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
 and the Greek economy
In-Reply-To: <CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>	<20150221032559.727D07A0792@apps0.cs.toronto.edu>	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>	<CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>
	<CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>
Message-ID: <54EAEFA8.4020101@osn.de>

Hi,

I remember there was a problem with the flow control settings in the ixgbe
driver, so I updated it a long time ago for our internal servers to 2.5.8.
Last weekend I integrated the latest changes from the FreeBSD driver to 
bring
the illumos ixgbe to 2.5.25 but I had no time to test it, so it's completely
untested!


If you would like to give the latest driver a try you can fetch the
kernel modules from https://cloud.osn.de/index.php/s/Fb4so9RsNnXA7r9

Clone your boot environment, place the modules in the new environment
and update the boot-archive of the new BE.

  - Joerg




On 23.02.2015 02:54, W Verb wrote:
> By the way, to those of you who have working setups: please send me
> your pool/volume settings, interface linkprops, and any kernel tuning
> parameters you may have set.
>
> Thanks,
> Warren V
>
> On Sat, Feb 21, 2015 at 7:59 AM, Schweiss, Chip <chip at innovates.com> wrote:
>> I can't say I totally agree with your performance assessment.   I run Intel
>> X520 in all my OmniOS boxes.
>>
>> Here is a capture of nfssvrtop I made while running many storage vMotions
>> between two OmniOS boxes hosting NFS datastores.   This is a 10 host VMware
>> cluster.  Both OmniOS boxes are dual 10G connected with copper twin-ax to
>> the in rack Nexus 5010.
>>
>> VMware does 100% sync writes, I use ZeusRAM SSDs for log devices.
>>
>> -Chip
>>
>> 2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB, swrite: 15985    KB,
>> awrite: 1875455  KB
>>
>> Ver     Client           NFSOPS   Reads SWrites AWrites Commits   Rd_bw
>> SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%
>>
>> 4       10.28.17.105          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.17.215          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.17.213          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       10.28.16.151          0       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 4       all                   1       0       0       0       0       0
>> 0       0       0       0       0       0       0
>>
>> 3       10.28.16.175          3       0       3       0       0       1
>> 11       0    4806      48       0       0      85
>>
>> 3       10.28.16.183          6       0       6       0       0       3
>> 162       0     549     124       0       0      73
>>
>> 3       10.28.16.180         11       0      10       0       0       3
>> 27       0     776      89       0       0      67
>>
>> 3       10.28.16.176         28       2      26       0       0      10
>> 405       0    2572     198       0       0     100
>>
>> 3       10.28.16.178       4606    4602       4       0       0  294534
>> 3       0     723      49       0       0      99
>>
>> 3       10.28.16.179       4905    4879      26       0       0  312208
>> 311       0     735     271       0       0      99
>>
>> 3       10.28.16.181       5515    5502      13       0       0  352107
>> 77       0      89      87       0       0      99
>>
>> 3       10.28.16.184      12095   12059      10       0       0  763014
>> 39       0     249     147       0       0      99
>>
>> 3       10.28.58.1        15401    6040     116    6354      53  191605
>> 474  202346     192      96     144      83      99
>>
>> 3       all               42574   33086     217    6354      53 1913488
>> 1582  202300     348     138     153     105      99
>>
>>
>>
>>
>>
>> On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com> wrote:
>>>
>>> Hello All,
>>>
>>> Thank you for your replies.
>>> I tried a few things, and found the following:
>>>
>>> 1: Disabling hyperthreading support in the BIOS drops performance overall
>>> by a factor of 4.
>>> 2: Disabling VT support also seems to have some effect, although it
>>> appears to be minor. But this has the amusing side effect of fixing the
>>> hangs I've been experiencing with fast reboot. Probably by disabling kvm.
>>> 3: The performance tests are a bit tricky to quantify because of caching
>>> effects. In fact, I'm not entirely sure what is happening here. It's just
>>> best to describe what I'm seeing:
>>>
>>> The commands I'm using to test are
>>> dd if=/dev/zero of=./test.dd bs=2M count=5000
>>> dd of=/dev/null if=./test.dd bs=2M count=5000
>>> The host vm is running Centos 6.6, and has the latest vmtools installed.
>>> There is a host cache on an SSD local to the host that is also in place.
>>> Disabling the host cache didn't immediately have an effect as far as I could
>>> see.
>>>
>>> The host MTU set to 3000 on all iSCSI interfaces for all tests.
>>>
>>> Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
>>> yields an average speed over three tests of 137MB/s. The read test yields an
>>> average over three tests of 5MB/s.
>>>
>>> Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
>>> 140MB/s, and the read tests yield 53MB/s. It's important to note here that
>>> if I cut the read test short at only 2-3GB, I get results upwards of
>>> 350MB/s, which I assume is local cache-related distortion.
>>>
>>> Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield
>>> about 142MB/s.
>>> Test 4: MTU of 1000: Read test at 182MB/s.
>>> Test 5: MTU of 900: Read test at 130 MB/s.
>>> Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now
>>> consistently at about 300MB/s.
>>> Test 7: MTU of 1200: Read test at 124MB/s.
>>> Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>>>
>>> A few final notes:
>>> L1ARC grabs about 10GB of RAM during the tests, so there's definitely some
>>> read caching going on.
>>> The write operations are easier to observe with iostat, and I'm seeing io
>>> rates that closely correlate with the network write speeds.
>>>
>>>
>>> Chris, thanks for your specific details. I'd appreciate it if you could
>>> tell me which copper NIC you tried, as well as to pass on the iSCSI tuning
>>> parameters.
>>>
>>> I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead of
>>> the 82599 in the X520. If I get similar results with my fiber transcievers,
>>> I'll see if I can get a hold of copper ones.
>>>
>>> But I should mention that I did indeed look at PHY/MAC error rates, and
>>> they are nil.
>>>
>>> -Warren V
>>>
>>> On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
>>> wrote:
>>>>
>>>>> After installation and configuration, I observed all kinds of bad
>>>>> behavior
>>>>> in the network traffic between the hosts and the server. All of this
>>>>> bad
>>>>> behavior is traced to the ixgbe driver on the storage server. Without
>>>>> going
>>>>> into the full troubleshooting process, here are my takeaways:
>>>> [...]
>>>>
>>>>   For what it's worth, we managed to achieve much better line rates on
>>>> copper 10G ixgbe hardware of various descriptions between OmniOS
>>>> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
>>>> believe OmniOS could do TCP at full line rate but I think we managed 700+
>>>> Mbytes/sec on both transmit and receive and we got basically disk-limited
>>>> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
>>>> OmniOS iSCSI initiator, Linux iSCSI targets).
>>>>
>>>>   I don't believe we did any specific kernel tuning (and in fact some of
>>>> our attempts to fiddle ixgbe driver parameters blew up in our face).
>>>> We did tune iSCSI connection parameters to increase various buffer
>>>> sizes so that ZFS could do even large single operations in single iSCSI
>>>> transactions. (More details available if people are interested.)
>>>>
>>>>> 10: At the wire level, the speed problems are clearly due to pauses in
>>>>> response time by omnios. At 9000 byte frame sizes, I see a good number
>>>>> of duplicate ACKs and fast retransmits during read operations (when
>>>>> omnios is transmitting). But below about a 4100-byte MTU on omnios
>>>>> (which seems to correlate to 4096-byte iSCSI block transfers), the
>>>>> transmission errors fade away and we only see the transmission pause
>>>>> problem.
>>>>
>>>>   This is what really attracted my attention. In our OmniOS setup, our
>>>> specific Intel hardware had ixgbe driver issues that could cause
>>>> activity stalls during once-a-second link heartbeat checks. This
>>>> obviously had an effect at the TCP and iSCSI layers. My initial message
>>>> to illumos-developer sparked a potentially interesting discussion:
>>>>
>>>>
>>>> http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/
>>>>
>>>> If you think this is a possibility in your setup, I've put the DTrace
>>>> script I used to hunt for this up on the web:
>>>>
>>>>          http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>>>>
>>>> This isn't the only potential source of driver stalls by any means, it's
>>>> just the one I found. You may also want to look at lockstat in general,
>>>> as information it reported is what led us to look specifically at the
>>>> ixgbe code here.
>>>>
>>>> (If you suspect kernel/driver issues, lockstat combined with kernel
>>>> source is a really excellent resource.)
>>>>
>>>>          - cks
>>>
>>>
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

-- 
OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg
Tel: +49 911 39905-0 - Fax: +49 911 39905-55 - http://www.osn.de
HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann

From wverb73 at gmail.com  Mon Feb 23 09:31:55 2015
From: wverb73 at gmail.com (W Verb)
Date: Mon, 23 Feb 2015 01:31:55 -0800
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: <54EAEFA8.4020101@osn.de>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>
	<20150221032559.727D07A0792@apps0.cs.toronto.edu>
	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
	<CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>
	<CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>
	<54EAEFA8.4020101@osn.de>
Message-ID: <CAHN_Y25MXSsdNzJMvDAsuerJ0Tp74oV4CJjtac85vxmYg8LcXg@mail.gmail.com>

Thank you Joerg,

I've downloaded the package and will try it tomorrow.

The only thing I can add at this point is that upon review of my testing, I
may have performed my "pkg -u" between the initial quad-gig performance
test and installing the 10G NIC. So this may be a new problem introduced in
the latest updates.

Those of you who are running 10G and have not upgraded to the latest
kernel, etc, might want to do some additional testing before running the
update.

-Warren V

On Mon, Feb 23, 2015 at 1:15 AM, Joerg Goltermann <jg at osn.de> wrote:

> Hi,
>
> I remember there was a problem with the flow control settings in the ixgbe
> driver, so I updated it a long time ago for our internal servers to 2.5.8.
> Last weekend I integrated the latest changes from the FreeBSD driver to
> bring
> the illumos ixgbe to 2.5.25 but I had no time to test it, so it's
> completely
> untested!
>
>
> If you would like to give the latest driver a try you can fetch the
> kernel modules from https://cloud.osn.de/index.php/s/Fb4so9RsNnXA7r9
>
> Clone your boot environment, place the modules in the new environment
> and update the boot-archive of the new BE.
>
>  - Joerg
>
>
>
>
>
> On 23.02.2015 02:54, W Verb wrote:
>
>> By the way, to those of you who have working setups: please send me
>> your pool/volume settings, interface linkprops, and any kernel tuning
>> parameters you may have set.
>>
>> Thanks,
>> Warren V
>>
>> On Sat, Feb 21, 2015 at 7:59 AM, Schweiss, Chip <chip at innovates.com>
>> wrote:
>>
>>> I can't say I totally agree with your performance assessment.   I run
>>> Intel
>>> X520 in all my OmniOS boxes.
>>>
>>> Here is a capture of nfssvrtop I made while running many storage vMotions
>>> between two OmniOS boxes hosting NFS datastores.   This is a 10 host
>>> VMware
>>> cluster.  Both OmniOS boxes are dual 10G connected with copper twin-ax to
>>> the in rack Nexus 5010.
>>>
>>> VMware does 100% sync writes, I use ZeusRAM SSDs for log devices.
>>>
>>> -Chip
>>>
>>> 2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB, swrite: 15985
>>> KB,
>>> awrite: 1875455  KB
>>>
>>> Ver     Client           NFSOPS   Reads SWrites AWrites Commits   Rd_bw
>>> SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%
>>>
>>> 4       10.28.17.105          0       0       0       0       0       0
>>> 0       0       0       0       0       0       0
>>>
>>> 4       10.28.17.215          0       0       0       0       0       0
>>> 0       0       0       0       0       0       0
>>>
>>> 4       10.28.17.213          0       0       0       0       0       0
>>> 0       0       0       0       0       0       0
>>>
>>> 4       10.28.16.151          0       0       0       0       0       0
>>> 0       0       0       0       0       0       0
>>>
>>> 4       all                   1       0       0       0       0       0
>>> 0       0       0       0       0       0       0
>>>
>>> 3       10.28.16.175          3       0       3       0       0       1
>>> 11       0    4806      48       0       0      85
>>>
>>> 3       10.28.16.183          6       0       6       0       0       3
>>> 162       0     549     124       0       0      73
>>>
>>> 3       10.28.16.180         11       0      10       0       0       3
>>> 27       0     776      89       0       0      67
>>>
>>> 3       10.28.16.176         28       2      26       0       0      10
>>> 405       0    2572     198       0       0     100
>>>
>>> 3       10.28.16.178       4606    4602       4       0       0  294534
>>> 3       0     723      49       0       0      99
>>>
>>> 3       10.28.16.179       4905    4879      26       0       0  312208
>>> 311       0     735     271       0       0      99
>>>
>>> 3       10.28.16.181       5515    5502      13       0       0  352107
>>> 77       0      89      87       0       0      99
>>>
>>> 3       10.28.16.184      12095   12059      10       0       0  763014
>>> 39       0     249     147       0       0      99
>>>
>>> 3       10.28.58.1        15401    6040     116    6354      53  191605
>>> 474  202346     192      96     144      83      99
>>>
>>> 3       all               42574 33086     217    6354      53 1913488
>>> 1582  202300     348     138     153     105      99
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com> wrote:
>>>
>>>>
>>>> Hello All,
>>>>
>>>> Thank you for your replies.
>>>> I tried a few things, and found the following:
>>>>
>>>> 1: Disabling hyperthreading support in the BIOS drops performance
>>>> overall
>>>> by a factor of 4.
>>>> 2: Disabling VT support also seems to have some effect, although it
>>>> appears to be minor. But this has the amusing side effect of fixing the
>>>> hangs I've been experiencing with fast reboot. Probably by disabling
>>>> kvm.
>>>> 3: The performance tests are a bit tricky to quantify because of caching
>>>> effects. In fact, I'm not entirely sure what is happening here. It's
>>>> just
>>>> best to describe what I'm seeing:
>>>>
>>>> The commands I'm using to test are
>>>> dd if=/dev/zero of=./test.dd bs=2M count=5000
>>>> dd of=/dev/null if=./test.dd bs=2M count=5000
>>>> The host vm is running Centos 6.6, and has the latest vmtools installed.
>>>> There is a host cache on an SSD local to the host that is also in place.
>>>> Disabling the host cache didn't immediately have an effect as far as I
>>>> could
>>>> see.
>>>>
>>>> The host MTU set to 3000 on all iSCSI interfaces for all tests.
>>>>
>>>> Test 1: Right after reboot, with an ixgbe MTU of 9000, the write test
>>>> yields an average speed over three tests of 137MB/s. The read test
>>>> yields an
>>>> average over three tests of 5MB/s.
>>>>
>>>> Test 2: After setting "ifconfig ixgbe0 mtu 3000", the write tests yield
>>>> 140MB/s, and the read tests yield 53MB/s. It's important to note here
>>>> that
>>>> if I cut the read test short at only 2-3GB, I get results upwards of
>>>> 350MB/s, which I assume is local cache-related distortion.
>>>>
>>>> Test 3: MTU of 1500. Read tests are up to 156 MB/s. Write tests yield
>>>> about 142MB/s.
>>>> Test 4: MTU of 1000: Read test at 182MB/s.
>>>> Test 5: MTU of 900: Read test at 130 MB/s.
>>>> Test 6: MTU of 1000: Read test at 160MB/s. Write tests are now
>>>> consistently at about 300MB/s.
>>>> Test 7: MTU of 1200: Read test at 124MB/s.
>>>> Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>>>>
>>>> A few final notes:
>>>> L1ARC grabs about 10GB of RAM during the tests, so there's definitely
>>>> some
>>>> read caching going on.
>>>> The write operations are easier to observe with iostat, and I'm seeing
>>>> io
>>>> rates that closely correlate with the network write speeds.
>>>>
>>>>
>>>> Chris, thanks for your specific details. I'd appreciate it if you could
>>>> tell me which copper NIC you tried, as well as to pass on the iSCSI
>>>> tuning
>>>> parameters.
>>>>
>>>> I've ordered an Intel EXPX9502AFXSR, which uses the 82598 chip instead
>>>> of
>>>> the 82599 in the X520. If I get similar results with my fiber
>>>> transcievers,
>>>> I'll see if I can get a hold of copper ones.
>>>>
>>>> But I should mention that I did indeed look at PHY/MAC error rates, and
>>>> they are nil.
>>>>
>>>> -Warren V
>>>>
>>>> On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann <cks at cs.toronto.edu>
>>>> wrote:
>>>>
>>>>>
>>>>>  After installation and configuration, I observed all kinds of bad
>>>>>> behavior
>>>>>> in the network traffic between the hosts and the server. All of this
>>>>>> bad
>>>>>> behavior is traced to the ixgbe driver on the storage server. Without
>>>>>> going
>>>>>> into the full troubleshooting process, here are my takeaways:
>>>>>>
>>>>> [...]
>>>>>
>>>>>   For what it's worth, we managed to achieve much better line rates on
>>>>> copper 10G ixgbe hardware of various descriptions between OmniOS
>>>>> and CentOS 7 (I don't think we ever tested OmniOS to OmniOS). I don't
>>>>> believe OmniOS could do TCP at full line rate but I think we managed
>>>>> 700+
>>>>> Mbytes/sec on both transmit and receive and we got basically
>>>>> disk-limited
>>>>> speeds with iSCSI (across multiple disks on multi-disk mirrored pools,
>>>>> OmniOS iSCSI initiator, Linux iSCSI targets).
>>>>>
>>>>>   I don't believe we did any specific kernel tuning (and in fact some
>>>>> of
>>>>> our attempts to fiddle ixgbe driver parameters blew up in our face).
>>>>> We did tune iSCSI connection parameters to increase various buffer
>>>>> sizes so that ZFS could do even large single operations in single iSCSI
>>>>> transactions. (More details available if people are interested.)
>>>>>
>>>>>  10: At the wire level, the speed problems are clearly due to pauses in
>>>>>> response time by omnios. At 9000 byte frame sizes, I see a good number
>>>>>> of duplicate ACKs and fast retransmits during read operations (when
>>>>>> omnios is transmitting). But below about a 4100-byte MTU on omnios
>>>>>> (which seems to correlate to 4096-byte iSCSI block transfers), the
>>>>>> transmission errors fade away and we only see the transmission pause
>>>>>> problem.
>>>>>>
>>>>>
>>>>>   This is what really attracted my attention. In our OmniOS setup, our
>>>>> specific Intel hardware had ixgbe driver issues that could cause
>>>>> activity stalls during once-a-second link heartbeat checks. This
>>>>> obviously had an effect at the TCP and iSCSI layers. My initial message
>>>>> to illumos-developer sparked a potentially interesting discussion:
>>>>>
>>>>>
>>>>> http://www.listbox.com/member/archive/182179/2014/10/sort/
>>>>> time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-
>>>>> 11E4-A39C-D534381BA44D/
>>>>>
>>>>> If you think this is a possibility in your setup, I've put the DTrace
>>>>> script I used to hunt for this up on the web:
>>>>>
>>>>>          http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d
>>>>>
>>>>> This isn't the only potential source of driver stalls by any means,
>>>>> it's
>>>>> just the one I found. You may also want to look at lockstat in general,
>>>>> as information it reported is what led us to look specifically at the
>>>>> ixgbe code here.
>>>>>
>>>>> (If you suspect kernel/driver issues, lockstat combined with kernel
>>>>> source is a really excellent resource.)
>>>>>
>>>>>          - cks
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OmniOS-discuss mailing list
>>>> OmniOS-discuss at lists.omniti.com
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>>
>>>>
>>>  _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
> --
> OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg
> Tel: +49 911 39905-0 - Fax: +49 911 39905-55 - http://www.osn.de
> HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150223/308c6d99/attachment-0001.html>

From danmcd at omniti.com  Mon Feb 23 16:06:15 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 23 Feb 2015 11:06:15 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
	OmniOS r151010
In-Reply-To: <54E7B1D8.3070009@gmx.li>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
	<E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>
	<54E7B1D8.3070009@gmx.li>
Message-ID: <07CAF7F6-D55E-47AB-AF40-96B717B03106@omniti.com>


> On Feb 20, 2015, at 5:14 PM, Dominik Hassler <hasslerd at gmx.li> wrote:
> 
> Dan,
> 
> I had a look at HVM-807 you mentioned.
> 
> Shouldn't line 251 in 'net/vnic.c' be
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX); i++, iov++) {
> 
> instead of
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX - 1); i++, iov++) {
> 
> As otherwise the last element of the frameio will not be used at all if
> iovcnt is greater than the frameio buffer size?

I've no idea, honestly. I count on upstream (Joyent) to look at all of that.  If you've found a genuine problem in illumos-kvm or illumos-kvm-cmd, perhaps pinging smartos-discuss, or some other appropriate list may be your best bet.

Dan


From jg at osn.de  Mon Feb 23 16:21:38 2015
From: jg at osn.de (Joerg Goltermann)
Date: Mon, 23 Feb 2015 17:21:38 +0100
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
 and the Greek economy
In-Reply-To: <CAHN_Y25MXSsdNzJMvDAsuerJ0Tp74oV4CJjtac85vxmYg8LcXg@mail.gmail.com>
References: <CAHN_Y258bM42G6bCXpMv0fd4s5JMHDY7MBc7oTTnPCp6yN+a5Q@mail.gmail.com>	<20150221032559.727D07A0792@apps0.cs.toronto.edu>	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>	<CALeZrrRvQtRoSX16YJX7RmZX3C6+E4w9z6RBG_xPhJzdH5E0bg@mail.gmail.com>	<CAHN_Y26z=mPjPEJoV2G19Mr0uscFO8f1U-CwpDODGoE5y=vgUg@mail.gmail.com>	<54EAEFA8.4020101@osn.de>
	<CAHN_Y25MXSsdNzJMvDAsuerJ0Tp74oV4CJjtac85vxmYg8LcXg@mail.gmail.com>
Message-ID: <54EB5392.6030900@osn.de>

Hi,

I think your problem is caused by your link properties or your
switch settings. In general the standard ixgbe seems to perform
well.

I had trouble after changing the default flow control settings to "bi"
and this was my motivation to update the ixgbe driver a long time ago.
After I have updated our systems to ixgbe 2.5.8 I never had any
problems ....

Make sure your switch has support for jumbo frames and you use
the same mtu on all ports, otherwise the smallest will be used.

What switch do you use? I can tell you nice horror stories about
different vendors....

  - Joerg

On 23.02.2015 10:31, W Verb wrote:
> Thank you Joerg,
>
> I've downloaded the package and will try it tomorrow.
>
> The only thing I can add at this point is that upon review of my
> testing, I may have performed my "pkg -u" between the initial quad-gig
> performance test and installing the 10G NIC. So this may be a new
> problem introduced in the latest updates.
>
> Those of you who are running 10G and have not upgraded to the latest
> kernel, etc, might want to do some additional testing before running the
> update.
>
> -Warren V
>
> On Mon, Feb 23, 2015 at 1:15 AM, Joerg Goltermann <jg at osn.de
> <mailto:jg at osn.de>> wrote:
>
>     Hi,
>
>     I remember there was a problem with the flow control settings in the
>     ixgbe
>     driver, so I updated it a long time ago for our internal servers to
>     2.5.8.
>     Last weekend I integrated the latest changes from the FreeBSD driver
>     to bring
>     the illumos ixgbe to 2.5.25 but I had no time to test it, so it's
>     completely
>     untested!
>
>
>     If you would like to give the latest driver a try you can fetch the
>     kernel modules from
>     https://cloud.osn.de/index.__php/s/Fb4so9RsNnXA7r9
>     <https://cloud.osn.de/index.php/s/Fb4so9RsNnXA7r9>
>
>     Clone your boot environment, place the modules in the new environment
>     and update the boot-archive of the new BE.
>
>       - Joerg
>
>
>
>
>
>     On 23.02.2015 02:54, W Verb wrote:
>
>         By the way, to those of you who have working setups: please send me
>         your pool/volume settings, interface linkprops, and any kernel
>         tuning
>         parameters you may have set.
>
>         Thanks,
>         Warren V
>
>         On Sat, Feb 21, 2015 at 7:59 AM, Schweiss, Chip
>         <chip at innovates.com <mailto:chip at innovates.com>> wrote:
>
>             I can't say I totally agree with your performance
>             assessment.   I run Intel
>             X520 in all my OmniOS boxes.
>
>             Here is a capture of nfssvrtop I made while running many
>             storage vMotions
>             between two OmniOS boxes hosting NFS datastores.   This is a
>             10 host VMware
>             cluster.  Both OmniOS boxes are dual 10G connected with
>             copper twin-ax to
>             the in rack Nexus 5010.
>
>             VMware does 100% sync writes, I use ZeusRAM SSDs for log
>             devices.
>
>             -Chip
>
>             2014 Apr 24 08:05:51, load: 12.64, read: 17330243 KB,
>             swrite: 15985    KB,
>             awrite: 1875455  KB
>
>             Ver     Client           NFSOPS   Reads SWrites AWrites
>             Commits   Rd_bw
>             SWr_bw  AWr_bw    Rd_t   SWr_t   AWr_t   Com_t  Align%
>
>             4       10.28.17.105          0       0       0       0
>               0       0
>             0       0       0       0       0       0       0
>
>             4       10.28.17.215          0       0       0       0
>               0       0
>             0       0       0       0       0       0       0
>
>             4       10.28.17.213          0       0       0       0
>               0       0
>             0       0       0       0       0       0       0
>
>             4       10.28.16.151          0       0       0       0
>               0       0
>             0       0       0       0       0       0       0
>
>             4       all                   1       0       0       0
>               0       0
>             0       0       0       0       0       0       0
>
>             3       10.28.16.175          3       0       3       0
>               0       1
>             11       0    4806      48       0       0      85
>
>             3       10.28.16.183          6       0       6       0
>               0       3
>             162       0     549     124       0       0      73
>
>             3       10.28.16.180         11       0      10       0
>               0       3
>             27       0     776      89       0       0      67
>
>             3       10.28.16.176         28       2      26       0
>               0      10
>             405       0    2572     198       0       0     100
>
>             3       10.28.16.178       4606    4602       4       0
>               0  294534
>             3       0     723      49       0       0      99
>
>             3       10.28.16.179       4905    4879      26       0
>               0  312208
>             311       0     735     271       0       0      99
>
>             3       10.28.16.181       5515    5502      13       0
>               0  352107
>             77       0      89      87       0       0      99
>
>             3       10.28.16.184      12095   12059      10       0
>               0  763014
>             39       0     249     147       0       0      99
>
>             3       10.28.58.1        15401    6040     116    6354
>             53  191605
>             474  202346     192      96     144      83      99
>
>             3       all 42574 33086 <tel:42574%20%20%2033086>     217
>             6354      53 1913488
>             1582  202300     348     138     153     105      99
>
>
>
>
>
>             On Fri, Feb 20, 2015 at 11:46 PM, W Verb <wverb73 at gmail.com
>             <mailto:wverb73 at gmail.com>> wrote:
>
>
>                 Hello All,
>
>                 Thank you for your replies.
>                 I tried a few things, and found the following:
>
>                 1: Disabling hyperthreading support in the BIOS drops
>                 performance overall
>                 by a factor of 4.
>                 2: Disabling VT support also seems to have some effect,
>                 although it
>                 appears to be minor. But this has the amusing side
>                 effect of fixing the
>                 hangs I've been experiencing with fast reboot. Probably
>                 by disabling kvm.
>                 3: The performance tests are a bit tricky to quantify
>                 because of caching
>                 effects. In fact, I'm not entirely sure what is
>                 happening here. It's just
>                 best to describe what I'm seeing:
>
>                 The commands I'm using to test are
>                 dd if=/dev/zero of=./test.dd bs=2M count=5000
>                 dd of=/dev/null if=./test.dd bs=2M count=5000
>                 The host vm is running Centos 6.6, and has the latest
>                 vmtools installed.
>                 There is a host cache on an SSD local to the host that
>                 is also in place.
>                 Disabling the host cache didn't immediately have an
>                 effect as far as I could
>                 see.
>
>                 The host MTU set to 3000 on all iSCSI interfaces for all
>                 tests.
>
>                 Test 1: Right after reboot, with an ixgbe MTU of 9000,
>                 the write test
>                 yields an average speed over three tests of 137MB/s. The
>                 read test yields an
>                 average over three tests of 5MB/s.
>
>                 Test 2: After setting "ifconfig ixgbe0 mtu 3000", the
>                 write tests yield
>                 140MB/s, and the read tests yield 53MB/s. It's important
>                 to note here that
>                 if I cut the read test short at only 2-3GB, I get
>                 results upwards of
>                 350MB/s, which I assume is local cache-related distortion.
>
>                 Test 3: MTU of 1500. Read tests are up to 156 MB/s.
>                 Write tests yield
>                 about 142MB/s.
>                 Test 4: MTU of 1000: Read test at 182MB/s.
>                 Test 5: MTU of 900: Read test at 130 MB/s.
>                 Test 6: MTU of 1000: Read test at 160MB/s. Write tests
>                 are now
>                 consistently at about 300MB/s.
>                 Test 7: MTU of 1200: Read test at 124MB/s.
>                 Test 8: MTU of 1000: Read test at 161MB/s. Write at 261MB/s.
>
>                 A few final notes:
>                 L1ARC grabs about 10GB of RAM during the tests, so
>                 there's definitely some
>                 read caching going on.
>                 The write operations are easier to observe with iostat,
>                 and I'm seeing io
>                 rates that closely correlate with the network write speeds.
>
>
>                 Chris, thanks for your specific details. I'd appreciate
>                 it if you could
>                 tell me which copper NIC you tried, as well as to pass
>                 on the iSCSI tuning
>                 parameters.
>
>                 I've ordered an Intel EXPX9502AFXSR, which uses the
>                 82598 chip instead of
>                 the 82599 in the X520. If I get similar results with my
>                 fiber transcievers,
>                 I'll see if I can get a hold of copper ones.
>
>                 But I should mention that I did indeed look at PHY/MAC
>                 error rates, and
>                 they are nil.
>
>                 -Warren V
>
>                 On Fri, Feb 20, 2015 at 7:25 PM, Chris Siebenmann
>                 <cks at cs.toronto.edu <mailto:cks at cs.toronto.edu>>
>                 wrote:
>
>
>                         After installation and configuration, I observed
>                         all kinds of bad
>                         behavior
>                         in the network traffic between the hosts and the
>                         server. All of this
>                         bad
>                         behavior is traced to the ixgbe driver on the
>                         storage server. Without
>                         going
>                         into the full troubleshooting process, here are
>                         my takeaways:
>
>                     [...]
>
>                        For what it's worth, we managed to achieve much
>                     better line rates on
>                     copper 10G ixgbe hardware of various descriptions
>                     between OmniOS
>                     and CentOS 7 (I don't think we ever tested OmniOS to
>                     OmniOS). I don't
>                     believe OmniOS could do TCP at full line rate but I
>                     think we managed 700+
>                     Mbytes/sec on both transmit and receive and we got
>                     basically disk-limited
>                     speeds with iSCSI (across multiple disks on
>                     multi-disk mirrored pools,
>                     OmniOS iSCSI initiator, Linux iSCSI targets).
>
>                        I don't believe we did any specific kernel tuning
>                     (and in fact some of
>                     our attempts to fiddle ixgbe driver parameters blew
>                     up in our face).
>                     We did tune iSCSI connection parameters to increase
>                     various buffer
>                     sizes so that ZFS could do even large single
>                     operations in single iSCSI
>                     transactions. (More details available if people are
>                     interested.)
>
>                         10: At the wire level, the speed problems are
>                         clearly due to pauses in
>                         response time by omnios. At 9000 byte frame
>                         sizes, I see a good number
>                         of duplicate ACKs and fast retransmits during
>                         read operations (when
>                         omnios is transmitting). But below about a
>                         4100-byte MTU on omnios
>                         (which seems to correlate to 4096-byte iSCSI
>                         block transfers), the
>                         transmission errors fade away and we only see
>                         the transmission pause
>                         problem.
>
>
>                        This is what really attracted my attention. In
>                     our OmniOS setup, our
>                     specific Intel hardware had ixgbe driver issues that
>                     could cause
>                     activity stalls during once-a-second link heartbeat
>                     checks. This
>                     obviously had an effect at the TCP and iSCSI layers.
>                     My initial message
>                     to illumos-developer sparked a potentially
>                     interesting discussion:
>
>
>                     http://www.listbox.com/member/__archive/182179/2014/10/sort/__time_rev/page/16/entry/6:405/__20141003125035:6357079A-4B1D-__11E4-A39C-D534381BA44D/
>                     <http://www.listbox.com/member/archive/182179/2014/10/sort/time_rev/page/16/entry/6:405/20141003125035:6357079A-4B1D-11E4-A39C-D534381BA44D/>
>
>                     If you think this is a possibility in your setup,
>                     I've put the DTrace
>                     script I used to hunt for this up on the web:
>
>                     http://www.cs.toronto.edu/~__cks/src/omnios-ixgbe/ixgbe___delay.d
>                     <http://www.cs.toronto.edu/~cks/src/omnios-ixgbe/ixgbe_delay.d>
>
>                     This isn't the only potential source of driver
>                     stalls by any means, it's
>                     just the one I found. You may also want to look at
>                     lockstat in general,
>                     as information it reported is what led us to look
>                     specifically at the
>                     ixgbe code here.
>
>                     (If you suspect kernel/driver issues, lockstat
>                     combined with kernel
>                     source is a really excellent resource.)
>
>                               - cks
>
>
>
>
>                 _________________________________________________
>                 OmniOS-discuss mailing list
>                 OmniOS-discuss at lists.omniti.__com
>                 <mailto:OmniOS-discuss at lists.omniti.com>
>                 http://lists.omniti.com/__mailman/listinfo/omnios-__discuss
>                 <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>
>
>         _________________________________________________
>         OmniOS-discuss mailing list
>         OmniOS-discuss at lists.omniti.__com
>         <mailto:OmniOS-discuss at lists.omniti.com>
>         http://lists.omniti.com/__mailman/listinfo/omnios-__discuss
>         <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>
>
>     --
>     OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg
>     Tel: +49 911 39905-0 <tel:%2B49%20911%2039905-0> - Fax: +49 911
>     39905-55 <tel:%2B49%20911%2039905-55> - http://www.osn.de
>     HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann
>
>

-- 
OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg
Tel: +49 911 39905-0 - Fax: +49 911 39905-55 - http://www.osn.de
HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann

From cks at cs.toronto.edu  Mon Feb 23 16:29:48 2015
From: cks at cs.toronto.edu (Chris Siebenmann)
Date: Mon, 23 Feb 2015 11:29:48 -0500
Subject: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan,
	and the Greek economy
In-Reply-To: Your message of Fri, 20 Feb 2015 21:46:51 -0800.
	<CAHN_Y24MBOmS94DbzhDojT3wgZ9FKU+FMh9NR8RQaGQYi+_COQ@mail.gmail.com>
Message-ID: <20150223162948.F3F777A082C@apps0.cs.toronto.edu>

> Chris, thanks for your specific details. I'd appreciate it if you
> could tell me which copper NIC you tried, as well as to pass on the
> iSCSI tuning parameters.

 Our copper NIC experience is with onboard X540-AT2 ports on SuperMicro
hardware (which have the guaranteed 10-20 msec lock hold) and dual-port
82599EB TN cards (which have some sort of driver/hardware failure under
load that eventually leads to 2-second lock holds). I can't recommend
either with the current driver; we had to revert to 1G networking in
order to get stable servers.

 The iSCSI parameter modifications we do, across both initiators and
targets, are:

	initialr2t		no
	firstburstlength	128k
	maxrecvdataseglen	128k		[only on Linux backends]
	maxxmitdataseglen	128k		[only on Linux backends]

The OmniOS initiator doesn't need tuning for more than the first two
parameters; on the Linux backends we tune up all four. My extended
thoughts on these tuning parameters and why we touch them can be found
here:

   http://utcc.utoronto.ca/~cks/space/blog/tech/UnderstandingiSCSIProtocol
   http://utcc.utoronto.ca/~cks/space/blog/tech/LikelyISCSITuning

The short version is that these parameters probably only make a small
difference but their overall goal is to do 128KB ZFS reads and writes
in single iSCSI operations (although they will be fragmented at the TCP
layer) and to do iSCSI writes without a back-and-forth delay between
initiator and target (that's 'initialr2t no').

 I think basically everyone should use InitialR2T set to no and in
fact that it should be the software default. These days only unusually
limited iSCSI targets should need it to be otherwise and they can change
their setting for it (initiator and target must both agree to it being
'yes', so either can veto it).

	- cks

From danmcd at omniti.com  Mon Feb 23 16:06:15 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 23 Feb 2015 11:06:15 -0500
Subject: [OmniOS-discuss] UPCOMING: End of Support Life approaching for
	OmniOS r151010
In-Reply-To: <54E7B1D8.3070009@gmx.li>
References: <75A4C8F1-0452-4B40-94A9-210D85516725@omniti.com>
	<alpine.DEB.2.10.1502202051410.14595@froburg.oetiker.ch>
	<E49AE557-5BE4-4AE7-92EA-8D22A3C6CA77@omniti.com>
	<54E7B1D8.3070009@gmx.li>
Message-ID: <07CAF7F6-D55E-47AB-AF40-96B717B03106@omniti.com>


> On Feb 20, 2015, at 5:14 PM, Dominik Hassler <hasslerd at gmx.li> wrote:
> 
> Dan,
> 
> I had a look at HVM-807 you mentioned.
> 
> Shouldn't line 251 in 'net/vnic.c' be
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX); i++, iov++) {
> 
> instead of
> for (i = 0; i < MIN(iovcnt, FRAMEIO_NVECS_MAX - 1); i++, iov++) {
> 
> As otherwise the last element of the frameio will not be used at all if
> iovcnt is greater than the frameio buffer size?

I've no idea, honestly. I count on upstream (Joyent) to look at all of that.  If you've found a genuine problem in illumos-kvm or illumos-kvm-cmd, perhaps pinging smartos-discuss, or some other appropriate list may be your best bet.

Dan


From steve at linuxsuite.org  Tue Feb 24 16:55:44 2015
From: steve at linuxsuite.org (steve at linuxsuite.org)
Date: Tue, 24 Feb 2015 11:55:44 -0500
Subject: [OmniOS-discuss] Same hardware show different cfgadm?
Message-ID: <ca445a06952c9685f0c8546705fe6bee.squirrel@emailmg.netfirms.com>


  Howdy!

          I have 2 identical zfs storage configurations consisting of
OmniOS v11 r151004
on

         Dell R710 server
         LSI - SAS9201-16e: attached to 16 bay Rackable JBOD
         LSI -  SAS9200-8e : attached to 45 bay Supermicro JBOD

       Things seem to work fine, but there are very different outputs
for cfgadm. There are 53 JBOD disks total with 10 raidz zpools + 3 hot
spares.
There are 5 system disks on the main Dell 710

All pools function ok. Disks are all 3TB SAS

 cfgadm shows very different information..

root at zfs-1:~# cfgadm
Ap_Id                          Type         Receptacle   Occupant    
Condition
c1                             scsi-bus     connected    configured   unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    configured   unknown
c6                             scsi-sas     connected    configured   unknown
c7                             scsi-sas     connected    configured   unknown
c8                             scsi-sas     connected    unconfigured unknown
[snip]

root at zfs-2:~# cfgadm
Ap_Id                          Type         Receptacle   Occupant    
Condition
c0                             scsi-sas     connected    configured   unknown
c1                             scsi-bus     connected    configured   unknown
c2                             scsi-sas     connected    unconfigured unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    configured   unknown
c5                             scsi-sas     connected    configured   unknown
[snip]


             A more complete listing is even stranger..  ( see below for
complete output )

root at zfs-1:~# cfgadm -al |grep dsk |wc -l
32

root at zfs-2:~# cfgadm -al|grep dsk|wc -l
53

     Why do some disks show up as

c5::w5000c50055c9a011,0        disk-path    connected    configured   unknown

and others as

c5::dsk/c5t5000CCA03E202671d0  disk         connected    configured   unknown


       However, format command sees all disks and can access them normally...

        Ideas? Does this matter? Why does cfgadm show such different configs?

          thanx - steve

root at zfs-1:~# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant    
Conditionc5::w5000c50055c9a011,0        disk-path    connected   
configured   unknown

c1                             scsi-bus     connected    configured   unknown
c1::dsk/c1t0d0                 disk         connected    configured   unknown
c1::dsk/c1t1d0                 disk         connected    configured   unknown
c1::dsk/c1t2d0                 disk         connected    configured   unknown
c1::dsk/c1t4d0                 disk         connected    configured   unknown
c1::dsk/c1t5d0                 disk         connected    configured   unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    configured   unknown
c4::dsk/c4t5000CCA01A7C4FE5d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA01A7CD8A1d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA01A7CDFB5d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA01A72D899d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA01A72DDDDd0  disk         connected    configured   unknown
c4::es/ses0                    ESI          connected    configured   unknown
c4::smp/expd0                  smp          connected    configured   unknown
c4::w5000c50041a6ab01,0        disk-path    connected    configured   unknown
c4::w5000c50041aafa81,0        disk-path    connected    configured   unknown
c4::w5000c50041ac4e3d,0        disk-path    connected    configured   unknown
c4::w5000c50041e9e081,0        disk-path    connected    configured   unknown
c4::w5000c50041f2f09d,0        disk-path    connected    configured   unknown
c4::w5000c50055a24fc1,0        disk-path    connected    configured   unknown
c4::w5000c50055a25a01,0        disk-path    connected    configured   unknown
c4::w5000c50055a25e99,0        disk-path    connected    configured   unknown
c4::w5000c50055a259b9,0        disk-path    connected    configured   unknown
c4::w5000c50055c9bed5,0        disk-path    connected    unconfigured unknown
c4::w5000c50055c66abd,0        disk-path    connected    configured   unknown
c4::w5000c50055ca227d,0        disk-path    connected    configured   unknown
c4::w5000c50055ca363d,0        disk-path    connected    unconfigured unknown
c4::w5000c50055e66c29,0        disk-path    connected    configured   unknown
c4::w5000c50055e75ad1,0        disk-path    connected    unconfigured unknown
c4::w5000c50055e7546d,0        disk-path    connected    configured   unknown
c4::w5000c500423b5e9d,0        disk-path    connected    unconfigured unknown
c4::w5000c500423da1bd,0        disk-path    connected    unconfigured unknown
c4::w5000c500423e977d,0        disk-path    connected    configured   unknown
c4::w5000c500424c5e29,0        disk-path    connected    configured   unknown
c4::w5000c5004198a2c5,0        disk-path    connected    configured   unknown
c6                             scsi-sas     connected    configured   unknown
c6::dsk/c6t5000CCA03E1E5FF5d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E1F33D9d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E09BFB1d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E014A2Dd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E15B6F1d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E20AF2Dd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E20AF3Dd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E167AFDd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E167B7Dd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E167B29d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E172A05d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E210B6Dd0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E0626F1d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E012635d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E113475d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E126771d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E167559d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E167649d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E168005d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E168141d0  disk         connected    configured   unknown
c6::dsk/c6t5000CCA03E212455d0  disk         connected    configured   unknown
c6::es/ses1                    ESI          connected    configured   unknown
c6::smp/expd1                  smp          connected    configured   unknown
c7                             scsi-sas     connected    configured   unknown
c7::dsk/c7t5000CCA03E2126E1d0  disk         connected    configured   unknown
c7::es/ses2                    ESI          connected    configured   unknown
c7::smp/expd2                  smp          connected    configured   unknown
c7::w5000c50055c9bed5,0        disk-path    connected    configured   unknown
c7::w5000c50055ca39e9,0        disk-path    connected    configured   unknown
c7::w5000c50055ca363d,0        disk-path    connected    configured   unknown
c7::w5000c50055e7b4d5,0        disk-path    connected    unconfigured unknown
c7::w5000c50055e75ad1,0        disk-path    connected    configured   unknown
c7::w5000c50055e77a25,0        disk-path    connected    configured   unknown
c7::w5000c50055e75671,0        disk-path    connected    configured   unknown
c7::w5000c50055e75679,0        disk-path    connected    configured   unknown
c7::w5000c500423b5e9d,0        disk-path    connected    configured   unknown
c7::w5000c500423da1bd,0        disk-path    connected    configured   unknown
c7::w5000c500423dd3d9,0        disk-path    connected    configured   unknown
c8                             scsi-sas     connected    unconfigured unknown
usb8/1                         unknown      empty        unconfigured ok
usb8/2                         unknown      empty        unconfigured ok
usb9/1                         unknown      empty        unconfigured ok
usb9/2                         unknown      empty        unconfigured ok
usb10/1                        unknown      empty        unconfigured ok
usb10/2                        unknown      empty        unconfigured ok
usb10/3                        usb-hub      connected    configured   ok
usb10/3.1                      unknown      empty        unconfigured ok
usb10/3.2                      unknown      empty        unconfigured ok
usb10/3.3                      unknown      empty        unconfigured ok
usb10/4                        unknown      empty        unconfigured ok
usb11/1                        unknown      empty        unconfigured ok
usb11/2                        usb-device   connected    configured   ok
usb12/1                        unknown      empty        unconfigured ok
usb12/2                        unknown      empty        unconfigured ok
usb13/1                        unknown      empty        unconfigured ok
usb13/2                        unknown      empty        unconfigured ok
usb13/3                        unknown      empty        unconfigured ok
usb13/4                        unknown      empty        unconfigured ok


root at zfs-2:~# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant    
Condition
c0                             scsi-sas     connected    configured   unknown
c0::dsk/c0t5000CCA01A7E89F5d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01A80A9CDd0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01A80AB65d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01A886801d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01ABC7339d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01ABC9445d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA01ABC9535d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA03E1AA175d0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA03E11FABDd0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA03E210E5Dd0  disk         connected    configured   unknown
c0::dsk/c0t5000CCA03E0622C5d0  disk         connected    configured   unknown
c0::es/ses2                    ESI          connected    configured   unknown
c0::smp/expd2                  smp          connected    configured   unknown
c1                             scsi-bus     connected    configured   unknown
c1::dsk/c1t0d0                 disk         connected    configured   unknown
c1::dsk/c1t1d0                 disk         connected    configured   unknown
c1::dsk/c1t3d0                 disk         connected    configured   unknown
c1::dsk/c1t4d0                 disk         connected    configured   unknown
c1::dsk/c1t5d0                 disk         connected    configured   unknown
c2                             scsi-sas     connected    unconfigured unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    configured   unknown
c4::dsk/c4t5000CCA01A58FD39d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E0B00B5d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E1C4401d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E1E4A51d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E1EFECDd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E0065B5d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167AF9d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167B45d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167BA9d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167C0Dd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167C49d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167CA1d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167D71d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E215DCDd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E1676E9d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E16708Dd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E16767Dd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E16807Dd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E21594Dd0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167269d0  disk         connected    configured   unknown
c4::dsk/c4t5000CCA03E167801d0  disk         connected    configured   unknown
c4::es/ses0                    ESI          connected    configured   unknown
c4::smp/expd0                  smp          connected    configured   unknown
c5                             scsi-sas     connected    configured   unknown
c5::dsk/c5t5000CCA01A7D4D85d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01A7E0499d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01A7F5F69d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01A80E98Dd0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01A807E75d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01AD9C2F9d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01AD331EDd0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01ADAF64Dd0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA01ADDCBF9d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E1FD499d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E19F571d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E00745Dd0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E202671d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E216269d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E227781d0  disk         connected    configured   unknown
c5::dsk/c5t5000CCA03E726699d0  disk         connected    configured   unknown
c5::es/ses1                    ESI          connected    configured   unknown
c5::smp/expd1                  smp          connected    configured   unknown
c5::w5000c50055c9a011,0        disk-path    connected    configured   unknown
c5::w5000c50055c9ffb9,0        disk-path    connected    configured   unknown
c5::w5000c50055e6a5c9,0        disk-path    connected    configured   unknown
c5::w5000c50055e6664d,0        disk-path    connected    configured   unknown
c5::w5000c5005596543d,0        disk-path    connected    configured   unknown
usb8/1                         unknown      empty        unconfigured ok
usb8/2                         unknown      empty        unconfigured ok
usb9/1                         unknown      empty        unconfigured ok
usb9/2                         unknown      empty        unconfigured ok
usb10/1                        unknown      empty        unconfigured ok
usb10/2                        unknown      empty        unconfigured ok
usb10/3                        usb-hub      connected    configured   ok
usb10/3.1                      unknown      empty        unconfigured ok
usb10/3.2                      unknown      empty        unconfigured ok
usb10/3.3                      unknown      empty        unconfigured ok
usb10/4                        unknown      empty        unconfigured ok
usb11/1                        unknown      empty        unconfigured ok
usb11/2                        usb-device   connected    configured   ok
usb12/1                        unknown      empty        unconfigured ok
usb12/2                        unknown      empty        unconfigured ok
usb13/1                        unknown      empty        unconfigured ok
usb13/2                        unknown      empty        unconfigured ok
usb13/3                        unknown      empty        unconfigured ok
usb13/4                        unknown      empty        unconfigured ok




From mark0x01 at gmail.com  Wed Feb 25 07:23:58 2015
From: mark0x01 at gmail.com (Mark)
Date: Wed, 25 Feb 2015 20:23:58 +1300
Subject: [OmniOS-discuss] Same hardware show different cfgadm?
In-Reply-To: <ca445a06952c9685f0c8546705fe6bee.squirrel@emailmg.netfirms.com>
References: <ca445a06952c9685f0c8546705fe6bee.squirrel@emailmg.netfirms.com>
Message-ID: <54ED788E.8040907@gmail.com>

Do you have single or dual path enclosures?

Are the disks SAS or Sata?

It looks like it is seeing some of the disks on two paths, but not all.


c4::w5000c50055ca363d,0        disk-path
c7::w5000c50055ca363d,0        disk-path


Mark.


On 25/02/2015 5:55 a.m., steve at linuxsuite.org wrote:
>
>    Howdy!
>
>            I have 2 identical zfs storage configurations consisting of
> OmniOS v11 r151004
> on
>
>           Dell R710 server
>           LSI - SAS9201-16e: attached to 16 bay Rackable JBOD
>           LSI -  SAS9200-8e : attached to 45 bay Supermicro JBOD
>
>         Things seem to work fine, but there are very different outputs
> for cfgadm. There are 53 JBOD disks total with 10 raidz zpools + 3 hot
> spares.
> There are 5 system disks on the main Dell 710
>
> All pools function ok. Disks are all 3TB SAS
>
>   cfgadm shows very different information..
>
> root at zfs-1:~# cfgadm
> Ap_Id                          Type         Receptacle   Occupant
> Condition
> c1                             scsi-bus     connected    configured   unknown
> c3                             scsi-sas     connected    unconfigured unknown
> c4                             scsi-sas     connected    configured   unknown
> c6                             scsi-sas     connected    configured   unknown
> c7                             scsi-sas     connected    configured   unknown
> c8                             scsi-sas     connected    unconfigured unknown
> [snip]
>
> root at zfs-2:~# cfgadm
> Ap_Id                          Type         Receptacle   Occupant
> Condition
> c0                             scsi-sas     connected    configured   unknown
> c1                             scsi-bus     connected    configured   unknown
> c2                             scsi-sas     connected    unconfigured unknown
> c3                             scsi-sas     connected    unconfigured unknown
> c4                             scsi-sas     connected    configured   unknown
> c5                             scsi-sas     connected    configured   unknown
> [snip]
>
>
>               A more complete listing is even stranger..  ( see below for
> complete output )
>
> root at zfs-1:~# cfgadm -al |grep dsk |wc -l
> 32
>
> root at zfs-2:~# cfgadm -al|grep dsk|wc -l
> 53
>
>       Why do some disks show up as
>
> c5::w5000c50055c9a011,0        disk-path    connected    configured   unknown
>
> and others as
>
> c5::dsk/c5t5000CCA03E202671d0  disk         connected    configured   unknown
>
>
>         However, format command sees all disks and can access them normally...
>
>          Ideas? Does this matter? Why does cfgadm show such different configs?
>
>            thanx - steve
>
> root at zfs-1:~# cfgadm -al
> Ap_Id                          Type         Receptacle   Occupant
> Conditionc5::w5000c50055c9a011,0        disk-path    connected
> configured   unknown
>
> c1                             scsi-bus     connected    configured   unknown
> c1::dsk/c1t0d0                 disk         connected    configured   unknown
> c1::dsk/c1t1d0                 disk         connected    configured   unknown
> c1::dsk/c1t2d0                 disk         connected    configured   unknown
> c1::dsk/c1t4d0                 disk         connected    configured   unknown
> c1::dsk/c1t5d0                 disk         connected    configured   unknown
> c3                             scsi-sas     connected    unconfigured unknown
> c4                             scsi-sas     connected    configured   unknown
> c4::dsk/c4t5000CCA01A7C4FE5d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA01A7CD8A1d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA01A7CDFB5d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA01A72D899d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA01A72DDDDd0  disk         connected    configured   unknown
> c4::es/ses0                    ESI          connected    configured   unknown
> c4::smp/expd0                  smp          connected    configured   unknown
> c4::w5000c50041a6ab01,0        disk-path    connected    configured   unknown
> c4::w5000c50041aafa81,0        disk-path    connected    configured   unknown
> c4::w5000c50041ac4e3d,0        disk-path    connected    configured   unknown
> c4::w5000c50041e9e081,0        disk-path    connected    configured   unknown
> c4::w5000c50041f2f09d,0        disk-path    connected    configured   unknown
> c4::w5000c50055a24fc1,0        disk-path    connected    configured   unknown
> c4::w5000c50055a25a01,0        disk-path    connected    configured   unknown
> c4::w5000c50055a25e99,0        disk-path    connected    configured   unknown
> c4::w5000c50055a259b9,0        disk-path    connected    configured   unknown
> c4::w5000c50055c9bed5,0        disk-path    connected    unconfigured unknown
> c4::w5000c50055c66abd,0        disk-path    connected    configured   unknown
> c4::w5000c50055ca227d,0        disk-path    connected    configured   unknown
> c4::w5000c50055ca363d,0        disk-path    connected    unconfigured unknown
> c4::w5000c50055e66c29,0        disk-path    connected    configured   unknown
> c4::w5000c50055e75ad1,0        disk-path    connected    unconfigured unknown
> c4::w5000c50055e7546d,0        disk-path    connected    configured   unknown
> c4::w5000c500423b5e9d,0        disk-path    connected    unconfigured unknown
> c4::w5000c500423da1bd,0        disk-path    connected    unconfigured unknown
> c4::w5000c500423e977d,0        disk-path    connected    configured   unknown
> c4::w5000c500424c5e29,0        disk-path    connected    configured   unknown
> c4::w5000c5004198a2c5,0        disk-path    connected    configured   unknown
> c6                             scsi-sas     connected    configured   unknown
> c6::dsk/c6t5000CCA03E1E5FF5d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E1F33D9d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E09BFB1d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E014A2Dd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E15B6F1d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E20AF2Dd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E20AF3Dd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E167AFDd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E167B7Dd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E167B29d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E172A05d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E210B6Dd0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E0626F1d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E012635d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E113475d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E126771d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E167559d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E167649d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E168005d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E168141d0  disk         connected    configured   unknown
> c6::dsk/c6t5000CCA03E212455d0  disk         connected    configured   unknown
> c6::es/ses1                    ESI          connected    configured   unknown
> c6::smp/expd1                  smp          connected    configured   unknown
> c7                             scsi-sas     connected    configured   unknown
> c7::dsk/c7t5000CCA03E2126E1d0  disk         connected    configured   unknown
> c7::es/ses2                    ESI          connected    configured   unknown
> c7::smp/expd2                  smp          connected    configured   unknown
> c7::w5000c50055c9bed5,0        disk-path    connected    configured   unknown
> c7::w5000c50055ca39e9,0        disk-path    connected    configured   unknown
> c7::w5000c50055ca363d,0        disk-path    connected    configured   unknown
> c7::w5000c50055e7b4d5,0        disk-path    connected    unconfigured unknown
> c7::w5000c50055e75ad1,0        disk-path    connected    configured   unknown
> c7::w5000c50055e77a25,0        disk-path    connected    configured   unknown
> c7::w5000c50055e75671,0        disk-path    connected    configured   unknown
> c7::w5000c50055e75679,0        disk-path    connected    configured   unknown
> c7::w5000c500423b5e9d,0        disk-path    connected    configured   unknown
> c7::w5000c500423da1bd,0        disk-path    connected    configured   unknown
> c7::w5000c500423dd3d9,0        disk-path    connected    configured   unknown
> c8                             scsi-sas     connected    unconfigured unknown
> usb8/1                         unknown      empty        unconfigured ok
> usb8/2                         unknown      empty        unconfigured ok
> usb9/1                         unknown      empty        unconfigured ok
> usb9/2                         unknown      empty        unconfigured ok
> usb10/1                        unknown      empty        unconfigured ok
> usb10/2                        unknown      empty        unconfigured ok
> usb10/3                        usb-hub      connected    configured   ok
> usb10/3.1                      unknown      empty        unconfigured ok
> usb10/3.2                      unknown      empty        unconfigured ok
> usb10/3.3                      unknown      empty        unconfigured ok
> usb10/4                        unknown      empty        unconfigured ok
> usb11/1                        unknown      empty        unconfigured ok
> usb11/2                        usb-device   connected    configured   ok
> usb12/1                        unknown      empty        unconfigured ok
> usb12/2                        unknown      empty        unconfigured ok
> usb13/1                        unknown      empty        unconfigured ok
> usb13/2                        unknown      empty        unconfigured ok
> usb13/3                        unknown      empty        unconfigured ok
> usb13/4                        unknown      empty        unconfigured ok
>
>
> root at zfs-2:~# cfgadm -al
> Ap_Id                          Type         Receptacle   Occupant
> Condition
> c0                             scsi-sas     connected    configured   unknown
> c0::dsk/c0t5000CCA01A7E89F5d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01A80A9CDd0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01A80AB65d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01A886801d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01ABC7339d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01ABC9445d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA01ABC9535d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA03E1AA175d0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA03E11FABDd0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA03E210E5Dd0  disk         connected    configured   unknown
> c0::dsk/c0t5000CCA03E0622C5d0  disk         connected    configured   unknown
> c0::es/ses2                    ESI          connected    configured   unknown
> c0::smp/expd2                  smp          connected    configured   unknown
> c1                             scsi-bus     connected    configured   unknown
> c1::dsk/c1t0d0                 disk         connected    configured   unknown
> c1::dsk/c1t1d0                 disk         connected    configured   unknown
> c1::dsk/c1t3d0                 disk         connected    configured   unknown
> c1::dsk/c1t4d0                 disk         connected    configured   unknown
> c1::dsk/c1t5d0                 disk         connected    configured   unknown
> c2                             scsi-sas     connected    unconfigured unknown
> c3                             scsi-sas     connected    unconfigured unknown
> c4                             scsi-sas     connected    configured   unknown
> c4::dsk/c4t5000CCA01A58FD39d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E0B00B5d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E1C4401d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E1E4A51d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E1EFECDd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E0065B5d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167AF9d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167B45d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167BA9d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167C0Dd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167C49d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167CA1d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167D71d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E215DCDd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E1676E9d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E16708Dd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E16767Dd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E16807Dd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E21594Dd0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167269d0  disk         connected    configured   unknown
> c4::dsk/c4t5000CCA03E167801d0  disk         connected    configured   unknown
> c4::es/ses0                    ESI          connected    configured   unknown
> c4::smp/expd0                  smp          connected    configured   unknown
> c5                             scsi-sas     connected    configured   unknown
> c5::dsk/c5t5000CCA01A7D4D85d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01A7E0499d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01A7F5F69d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01A80E98Dd0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01A807E75d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01AD9C2F9d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01AD331EDd0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01ADAF64Dd0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA01ADDCBF9d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E1FD499d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E19F571d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E00745Dd0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E202671d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E216269d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E227781d0  disk         connected    configured   unknown
> c5::dsk/c5t5000CCA03E726699d0  disk         connected    configured   unknown
> c5::es/ses1                    ESI          connected    configured   unknown
> c5::smp/expd1                  smp          connected    configured   unknown
> c5::w5000c50055c9a011,0        disk-path    connected    configured   unknown
> c5::w5000c50055c9ffb9,0        disk-path    connected    configured   unknown
> c5::w5000c50055e6a5c9,0        disk-path    connected    configured   unknown
> c5::w5000c50055e6664d,0        disk-path    connected    configured   unknown
> c5::w5000c5005596543d,0        disk-path    connected    configured   unknown
> usb8/1                         unknown      empty        unconfigured ok
> usb8/2                         unknown      empty        unconfigured ok
> usb9/1                         unknown      empty        unconfigured ok
> usb9/2                         unknown      empty        unconfigured ok
> usb10/1                        unknown      empty        unconfigured ok
> usb10/2                        unknown      empty        unconfigured ok
> usb10/3                        usb-hub      connected    configured   ok
> usb10/3.1                      unknown      empty        unconfigured ok
> usb10/3.2                      unknown      empty        unconfigured ok
> usb10/3.3                      unknown      empty        unconfigured ok
> usb10/4                        unknown      empty        unconfigured ok
> usb11/1                        unknown      empty        unconfigured ok
> usb11/2                        usb-device   connected    configured   ok
> usb12/1                        unknown      empty        unconfigured ok
> usb12/2                        unknown      empty        unconfigured ok
> usb13/1                        unknown      empty        unconfigured ok
> usb13/2                        unknown      empty        unconfigured ok
> usb13/3                        unknown      empty        unconfigured ok
> usb13/4                        unknown      empty        unconfigured ok
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>


From tobi at oetiker.ch  Wed Feb 25 23:17:35 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Thu, 26 Feb 2015 00:17:35 +0100 (CET)
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
Message-ID: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>

experts!

If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
512n like in the olden days, or use the new 4Kn.

I know ZFS can deal with both ...

So what would be your choice, and WHY?

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From mir at miras.org  Wed Feb 25 23:35:57 2015
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 26 Feb 2015 00:35:57 +0100
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
In-Reply-To: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
Message-ID: <20150226003557.2f534c44@sleipner.datanom.net>

On Thu, 26 Feb 2015 00:17:35 +0100 (CET)
Tobias Oetiker <tobi at oetiker.ch> wrote:

> experts!
> 
> If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
> 512n like in the olden days, or use the new 4Kn.
> 
> I know ZFS can deal with both ...
> 
> So what would be your choice, and WHY?
> 
I would try to analyze the expected workload. Writing in small chunks
points at 512n while large chunks points towards 4Kn. Small chunks on
4Kn will waste a lot of space. If all else is equal the performance
should be the same. IMHO.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
To have died once is enough.
		-- Publius Vergilius Maro (Virgil)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150226/3f8a700b/attachment.bin>

From richard.elling at richardelling.com  Wed Feb 25 23:39:19 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Wed, 25 Feb 2015 15:39:19 -0800
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
In-Reply-To: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
Message-ID: <40D943AB-D3F6-4FF0-AD32-E4F6A7706FF4@richardelling.com>


> On Feb 25, 2015, at 3:17 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> 
> experts!
> 
> If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
> 512n like in the olden days, or use the new 4Kn.
> 
> I know ZFS can deal with both ...
> 
> So what would be your choice, and WHY?

Better yet, what would be your requirements, then your choice, then why?
 -- richard

> 
> cheers
> tobi
> 
> -- 
> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From protonwrangler at gmail.com  Thu Feb 26 00:09:21 2015
From: protonwrangler at gmail.com (Warren Marts)
Date: Wed, 25 Feb 2015 17:09:21 -0700
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
In-Reply-To: <40D943AB-D3F6-4FF0-AD32-E4F6A7706FF4@richardelling.com>
References: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
	<40D943AB-D3F6-4FF0-AD32-E4F6A7706FF4@richardelling.com>
Message-ID: <CAN8fiVtWRbNKg6G7KibUsfu8mpSiH5DYWxQ7MGMgaDmHUstLQA@mail.gmail.com>

At some point sourcing true 512 B/sector disks will get more difficult and
expensive - unless you knew this filesystem is likely to see lots of small
writes and performance or space efficiency are critical I'd lean to the 4KB
sector disks.

On Wed, Feb 25, 2015 at 4:39 PM, Richard Elling <
richard.elling at richardelling.com> wrote:

>
> > On Feb 25, 2015, at 3:17 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> >
> > experts!
> >
> > If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
> > 512n like in the olden days, or use the new 4Kn.
> >
> > I know ZFS can deal with both ...
> >
> > So what would be your choice, and WHY?
>
> Better yet, what would be your requirements, then your choice, then why?
>  -- richard
>
> >
> > cheers
> > tobi
> >
> > --
> > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150225/319fd1cb/attachment.html>

From tobi at oetiker.ch  Thu Feb 26 07:34:54 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Thu, 26 Feb 2015 08:34:54 +0100 (CET)
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
In-Reply-To: <CAN8fiVtWRbNKg6G7KibUsfu8mpSiH5DYWxQ7MGMgaDmHUstLQA@mail.gmail.com>
References: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
	<40D943AB-D3F6-4FF0-AD32-E4F6A7706FF4@richardelling.com>
	<CAN8fiVtWRbNKg6G7KibUsfu8mpSiH5DYWxQ7MGMgaDmHUstLQA@mail.gmail.com>
Message-ID: <alpine.DEB.2.10.1502260830010.14595@froburg.oetiker.ch>

Hi Marts,

Yesterday Warren Marts wrote:

> At some point sourcing true 512 B/sector disks will get more difficult and
> expensive - unless you knew this filesystem is likely to see lots of small
> writes and performance or space efficiency are critical I'd lean to the 4KB
> sector disks.

Amazingly, the new He6 drives from HGST only seem to exist in an
512n variant ... while the also new He8 drives only come in 4Kn and
512e types ...


> On Wed, Feb 25, 2015 at 4:39 PM, Richard Elling <
> richard.elling at richardelling.com> wrote:
>
> >
> > > On Feb 25, 2015, at 3:17 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> > >
> > > experts!
> > >
> > > If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
> > > 512n like in the olden days, or use the new 4Kn.
> > >
> > > I know ZFS can deal with both ...
> > >
> > > So what would be your choice, and WHY?
> >
> > Better yet, what would be your requirements, then your choice, then why?
> >  -- richard
> >
> > >
> > > cheers
> > > tobi
> > >
> > > --
> > > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> > > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> > >
> > > _______________________________________________
> > > OmniOS-discuss mailing list
> > > OmniOS-discuss at lists.omniti.com
> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From tobi at oetiker.ch  Thu Feb 26 08:37:34 2015
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Thu, 26 Feb 2015 09:37:34 +0100 (CET)
Subject: [OmniOS-discuss] HGST 7K6000 for a RAIDZ2 pool
In-Reply-To: <alpine.DEB.2.10.1502260830010.14595@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1502260011020.14595@froburg.oetiker.ch>
	<40D943AB-D3F6-4FF0-AD32-E4F6A7706FF4@richardelling.com>
	<CAN8fiVtWRbNKg6G7KibUsfu8mpSiH5DYWxQ7MGMgaDmHUstLQA@mail.gmail.com>
	<alpine.DEB.2.10.1502260830010.14595@froburg.oetiker.ch>
Message-ID: <alpine.DEB.2.10.1502260934550.14595@froburg.oetiker.ch>

Today Tobias Oetiker wrote:

> Hi Marts,
>
> Yesterday Warren Marts wrote:
>
> > At some point sourcing true 512 B/sector disks will get more difficult and
> > expensive - unless you knew this filesystem is likely to see lots of small
> > writes and performance or space efficiency are critical I'd lean to the 4KB
> > sector disks.
>
> Amazingly, the new He6 drives from HGST only seem to exist in an
> 512n variant ... while the also new He8 drives only come in 4Kn and
> 512e types ...

also while on the subject, there is

https://github.com/zfsonlinux/zfs/issues/548
https://github.com/zfsonlinux/zfs/issues/1807

which would indicate that using 512n is probably a better choice in
an environment where we are using zvol a lot

cheers
tobi


>
>
> > On Wed, Feb 25, 2015 at 4:39 PM, Richard Elling <
> > richard.elling at richardelling.com> wrote:
> >
> > >
> > > > On Feb 25, 2015, at 3:17 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> > > >
> > > > experts!
> > > >
> > > > If you were to buy 6TB disks for a RAIDZ2 Pool, would you go for
> > > > 512n like in the olden days, or use the new 4Kn.
> > > >
> > > > I know ZFS can deal with both ...
> > > >
> > > > So what would be your choice, and WHY?
> > >
> > > Better yet, what would be your requirements, then your choice, then why?
> > >  -- richard
> > >
> > > >
> > > > cheers
> > > > tobi
> > > >
> > > > --
> > > > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> > > > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> > > >
> > > > _______________________________________________
> > > > OmniOS-discuss mailing list
> > > > OmniOS-discuss at lists.omniti.com
> > > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> > >
> > > _______________________________________________
> > > OmniOS-discuss mailing list
> > > OmniOS-discuss at lists.omniti.com
> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> > >
> >
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From omnios at citrus-it.net  Fri Feb 27 10:05:26 2015
From: omnios at citrus-it.net (Andy)
Date: Fri, 27 Feb 2015 10:05:26 +0000 (GMT)
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
Message-ID: <alpine.GSO.2.00.1502270958170.5310@areb.pvgehf-vg.arg>


On Wed, 18 Feb 2015, Andy wrote:
; Is anyone running, or can anyone recommend, ; a Supermicro server roughly
; equivalent to the Del R730xd, or give me an idea of what chipsets/HBAs
; etc. to choose or avoid for OmniOS?
;
; Any help appreciated,

Thanks for all of the responses. We've been researching and mulling it over
and the latest plan is to go for Dell R730 servers with a 16 x 2.5" SAS
backplane, Intel Ethernet I350 networking and the standard Dell PERC H730
SAS controller.

Dell now officially support IT mode with the latest firmware for the
PERC H730 and I've read some encouraging reports. If we experience problems
then there are options to flash standard LSI firmware or to buy different
cards but I'm hopeful that it will just work.

If we go ahead, I'll let you all know how it goes!

Thanks,

Andy

-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From danmcd at omniti.com  Fri Feb 27 15:54:36 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 10:54:36 -0500
Subject: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In-Reply-To: <alpine.GSO.2.00.1502270958170.5310@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.1502181328130.16159@areb.pvgehf-vg.arg>
	<alpine.GSO.2.00.1502270958170.5310@areb.pvgehf-vg.arg>
Message-ID: <81F2A38D-5298-4FB8-B0BB-24E4506D6040@omniti.com>


> On Feb 27, 2015, at 5:05 AM, Andy <omnios at citrus-it.net> wrote:
> 
> If we go ahead, I'll let you all know how it goes!

Please do that.  If you can zap a Dell Standard HBA out of HW-RAID and into a raw-disk controller, that'd be a HUGE WIN for illumos distros everywhere.

Dan


From danmcd at omniti.com  Fri Feb 27 16:20:50 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 11:20:50 -0500
Subject: [OmniOS-discuss] (From illumos) Request for e1000g and igb testing
	for latest parts
References: <54F09887.10003@joyent.com>
Message-ID: <3ED021A3-FCDB-4D15-8F35-6C4CB1605A6E@omniti.com>

Hello OmniOS fans!

Robert Mustacchi from Joyent has updated e1000g & igb with more modern code from Intel.

If you are an OmniOS person looking to try this out, and need help updating either your e1000g or igb, please let me know via the OmniOS discussion list, or via unicast mail.  The binaries he supplies below should just *DROP IN* on any OmniOS revision.  I'll be trying this out on 64-bit e1000g (for 82579LM or 85274L) in bloody and maybe 64-bit igb in r151012 (for I210) respectively.

NOTE:  Use the non-debug bits unless you've built an entire BE around a DEBUG illumos-omnios build.

My advice for those who want to try is:

	beadm create new-intel
	beadm mount new-intel /mnt
	<place appropriate new 64-bit binaries in /mnt/kernel/drv/amd64/{igb,e1000g}>
	beadm umount new-intel
	beadm activate new-intel
	reboot

You can and should re-activate the existing BE when you're done testing.

Happy Intel Ethernetting!
Dan


> Begin forwarded message:
> 
> Date: February 27, 2015 at 11:17:11 AM EST
> From: "Robert Mustacchi via illumos-developer" <developer at lists.illumos.org>
> To: illumos Developer <developer at lists.illumos.org>, "smartos-discuss at lists.smartos.org" <smartos-discuss at lists.smartos.org>
> Subject: Re: [developer] Request for e1000g and igb testing for latest parts
> Reply-To: "Robert Mustacchi" <rm at joyent.com>
> 
> On 2/27/15 7:51 , Robert Mustacchi via illumos-developer wrote:
>> Hi,
>> 
>> Various Broadwell and related hardware is using newer e1000g parts, I
>> took the opportunity to sync up our common code to get support for them
>> and some newer igb parts, being more I218 and I210 branded parts. As
>> such, I have a newer igb and e1000g driver and would appreciate some
>> broader community testing on both the older devices and if you happen to
>> have one of the previously unsupported devices, some of the newer ones.
>> 
>> If you test it, can you please let me know what part you're using. You
>> can get the information from prtconf -d.
>> 
>> Here's all the pieces:
>> 
>> e1000g i386:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv/e1000g
>> e1000g amd64:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv/amd64/e1000g
>> igb i386:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv/igb
>> igb amd64:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv/amd64/igb
>> 
>> smartos platform:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/smartos/platform-20150223T225455Z.tgz
>> smartos iso:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/smartos/platform-20150223T225455Z.iso
>> smartos usb:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/smartos/platform-20150223T225455Z.usb.bz2
>> 
>> sdc platform:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/sdc/platform-20150223T225545Z.tgz
>> 
>> webrev:
>> https://us-east.manta.joyent.com/rmustacc/public/webrevs/e1000g/index.html
>> 
>> patch:
>> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/e1000g.patch
> 
> It was kindly pointed out that I had provided debug-build drivers. Here
> are links to the non-debug variants:
> 
> e1000g i386:
> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv-nd/e1000g
> e1000g amd64:
> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv-nd/amd64/e1000g
> igb i386:
> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv-nd/igb
> igb amd64:
> https://us-east.manta.joyent.com/rmustacc/public/preview/i218/drv-nd/amd64/igb
> 
> Thanks again,
> Robert
> 
> 
> 
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182179/21175029-813097db
> Modify Your Subscription: https://www.listbox.com/member/?member_id=21175029&id_secret=21175029-471fe0d4
> Powered by Listbox: http://www.listbox.com


From danmcd at omniti.com  Fri Feb 27 21:27:18 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 16:27:18 -0500
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on their
	boxes?
Message-ID: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>

This is a serious question.  A recent illumos push mildly affects people who have, but in OmniOS's case, it's a mild change (not a horrible one).

I'd appreciate some on-list feedback about who does and doesn't change their /etc/profile or /etc/.login vs. what we ship.

The next bloody update (and r151014) will alter the default file, and because of IPS, it won't be replaced IF the file was altered from the stock configuration.  Fortunately for OmniOS, it's mostly a don't-care.

I appreciate your feedback!

Thanks,
Dan McD -- OmniOS Engineering


From peter.tribble at gmail.com  Fri Feb 27 21:49:19 2015
From: peter.tribble at gmail.com (Peter Tribble)
Date: Fri, 27 Feb 2015 21:49:19 +0000
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <CAEgYsbGCUufj2ZTr7wqdKytYn_hM551sV0YFoJyAyc9Q9=mDyg@mail.gmail.com>

On Fri, Feb 27, 2015 at 9:27 PM, Dan McDonald <danmcd at omniti.com> wrote:

> This is a serious question.  A recent illumos push mildly affects people
> who have, but in OmniOS's case, it's a mild change (not a horrible one).
>
> I'd appreciate some on-list feedback about who does and doesn't change
> their /etc/profile or /etc/.login vs. what we ship.
>

This isn't OmniOS specific, but every production Solaris box I've managed
for the last decade
or two has had one or both fixed (usually .login, because tcsh was the
default, now /etc/profile
for other shells). Primarily to eradicate the completely unnecessary calls
to quota and mail.

I've seen some horrible monstrosities perpetrated by commercial software or
well-meaning
power users. PowerPath has this habit of whacking its stuff onto
/etc/profile, that's the one
3rd-party application that I've seen on a routine basis.


> The next bloody update (and r151014) will alter the default file, and
> because of IPS, it won't be replaced IF the file was altered from the stock
> configuration.  Fortunately for OmniOS, it's mostly a don't-care.
>
> I appreciate your feedback!
>
> Thanks,
> Dan McD -- OmniOS Engineering
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>



-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150227/a7d6dace/attachment.html>

From gate03 at landcroft.co.uk  Fri Feb 27 22:04:15 2015
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Sat, 28 Feb 2015 08:04:15 +1000
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
 their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <20150228080415.4a268256@dickless.landy.net>

On Fri, 27 Feb 2015 16:27:18 -0500
Dan McDonald <danmcd at omniti.com> wrote:

> This is a serious question.  A recent illumos push mildly affects
> people who have, but in OmniOS's case, it's a mild change (not a
> horrible one).
> 
> I'd appreciate some on-list feedback about who does and doesn't
> change their /etc/profile or /etc/.login vs. what we ship.

I leave as stock, but I'm a home user really.

Michael.

From sjorge+ml at blackdot.be  Fri Feb 27 22:16:24 2015
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Fri, 27 Feb 2015 23:16:24 +0100
Subject: [OmniOS-discuss]
 =?utf-8?q?Who_alters_/etc/profile_or_/etc/=2Elog?=
 =?utf-8?q?in_on_their_boxes=3F?=
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <2bace3c6b17daa3e0d0415d3c3b770a6@blackdot.be>

I made one small change:

                if [ -e /etc/motd.generated ]; then
                         /bin/cat -s /etc/motd-*
                 else
                         /bin/cat -s /etc/motd
                 fi


motd-00-header -> header unique per server
motd-01-info -> file generated every 10min with some basic info like 
disk usage for /export/home,...
motd-02-footer -> symlink to /etc/motd

Made no further changes.

Regards

Jorge


On 2015-02-27 22:27, Dan McDonald wrote:
> This is a serious question.  A recent illumos push mildly affects
> people who have, but in OmniOS's case, it's a mild change (not a
> horrible one).
> 
> I'd appreciate some on-list feedback about who does and doesn't change
> their /etc/profile or /etc/.login vs. what we ship.
> 
> The next bloody update (and r151014) will alter the default file, and
> because of IPS, it won't be replaced IF the file was altered from the
> stock configuration.  Fortunately for OmniOS, it's mostly a
> don't-care.
> 
> I appreciate your feedback!
> 
> Thanks,
> Dan McD -- OmniOS Engineering
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From tim at multitalents.net  Fri Feb 27 22:36:39 2015
From: tim at multitalents.net (Tim Rice)
Date: Fri, 27 Feb 2015 14:36:39 -0800 (PST)
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
 their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <alpine.UW2.2.11.1502271431560.15845@server01.int.multitalents.net>

On Fri, 27 Feb 2015, Dan McDonald wrote:

> This is a serious question.  A recent illumos push mildly affects people 
> who have, but in OmniOS's case, it's a mild change (not a horrible one).
> 
> I'd appreciate some on-list feedback about who does and doesn't change 
> their /etc/profile or /etc/.login vs. what we ship.

I can't remember a distro that I didn't modify /etc/profile and
/etc/.login (or their equivilants).

> The next bloody update (and r151014) will alter the default file, and 
> because of IPS, it won't be replaced IF the file was altered from the 
> stock configuration.  Fortunately for OmniOS, it's mostly a don't-care.

Maybe IPS has a way to install it/them as <filename>.new if it/they
have been modified.

> I appreciate your feedback!
> 
> Thanks,
> Dan McD -- OmniOS Engineering
> 

-- 
Tim Rice				Multitalents
tim at multitalents.net



From danmcd at omniti.com  Fri Feb 27 22:45:48 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 17:45:48 -0500
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their boxes?
In-Reply-To: <alpine.UW2.2.11.1502271431560.15845@server01.int.multitalents.net>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
	<alpine.UW2.2.11.1502271431560.15845@server01.int.multitalents.net>
Message-ID: <711514CD-8A85-4B1F-8AC5-A8DD1521585D@omniti.com>


> On Feb 27, 2015, at 5:36 PM, Tim Rice <tim at multitalents.net> wrote:
> 
>> The next bloody update (and r151014) will alter the default file, and 
>> because of IPS, it won't be replaced IF the file was altered from the 
>> stock configuration.  Fortunately for OmniOS, it's mostly a don't-care.
> 
> Maybe IPS has a way to install it/them as <filename>.new if it/they
> have been modified.

It does, and it will.  A profile.new will be installed if you have an altered /etc/profile.  Sorry I didn't mention that earlier.

Dan


From danmcd at omniti.com  Fri Feb 27 23:00:00 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 18:00:00 -0500
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <6CCB52C9-3873-4BB6-A071-72AFB879845F@omniti.com>


> On Feb 27, 2015, at 4:27 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> The next bloody update (and r151014) will alter the default file, and because of IPS, it won't be replaced IF the file was altered from the stock configuration.  Fortunately for OmniOS, it's mostly a don't-care.
> 
> I appreciate your feedback!

The change, BTW, is that upstream has ALSO removed machid(1) from illumos.  Since we already had done it, our diffs are  functionally the same, but I chose to inherit upstream's (modulo one copyright line) for future goodness:

bloody(~)[1]% diff -u /mnt/etc/profile /etc
--- /mnt/etc/profile    Tue Feb  3 15:01:21 2015
+++ /etc/profile        Thu Feb 26 23:24:53 2015
@@ -21,6 +21,7 @@
 # Copyright 2010 Sun Microsystems, Inc.  All rights reserved.
 # Use is subject to license terms.
 # Copyright 2012 OmniTI Computer Consulting, Inc.  All rights reserved.
+# Copyright 2015 Nexenta Systems, Inc. All rights reserved.
 #
 
 # The profile that all logins get before using their own .profile.
@@ -30,9 +31,7 @@
 
 if [ "$TERM" = "" ]
 then
-       CAN_I386=`/usr/bin/isalist | grep i386`
-       if [ -n "$CAN_I386" ]
-       then
+       if [ `uname -p` = "i386" ]; then
                TERM=sun-color
        else
                TERM=sun
bloody(~)[1]% 


So those of you with altered /etc/profile or /etc/.login files will need to be aware you'll be getting .new ones installed on the next bloody, or with r151014 if you don't do bloody.

FYI!
Dan


From sjorge+ml at blackdot.be  Fri Feb 27 23:02:12 2015
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Sat, 28 Feb 2015 00:02:12 +0100
Subject: [OmniOS-discuss]
 =?utf-8?q?Who_alters_/etc/profile_or_/etc/=2Elog?=
 =?utf-8?q?in_on_their_boxes=3F?=
In-Reply-To: <6CCB52C9-3873-4BB6-A071-72AFB879845F@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
	<6CCB52C9-3873-4BB6-A071-72AFB879845F@omniti.com>
Message-ID: <7dd8558e38d7e3d14857227692fcf699@blackdot.be>

Add a note to the release notes when the time comes, should be good 
enough?



On 2015-02-28 00:00, Dan McDonald wrote:
>> On Feb 27, 2015, at 4:27 PM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> The next bloody update (and r151014) will alter the default file, and 
>> because of IPS, it won't be replaced IF the file was altered from the 
>> stock configuration.  Fortunately for OmniOS, it's mostly a 
>> don't-care.
>> 
>> I appreciate your feedback!
> 
> The change, BTW, is that upstream has ALSO removed machid(1) from
> illumos.  Since we already had done it, our diffs are  functionally
> the same, but I chose to inherit upstream's (modulo one copyright
> line) for future goodness:
> 
> bloody(~)[1]% diff -u /mnt/etc/profile /etc
> --- /mnt/etc/profile    Tue Feb  3 15:01:21 2015
> +++ /etc/profile        Thu Feb 26 23:24:53 2015
> @@ -21,6 +21,7 @@
>  # Copyright 2010 Sun Microsystems, Inc.  All rights reserved.
>  # Use is subject to license terms.
>  # Copyright 2012 OmniTI Computer Consulting, Inc.  All rights 
> reserved.
> +# Copyright 2015 Nexenta Systems, Inc. All rights reserved.
>  #
> 
>  # The profile that all logins get before using their own .profile.
> @@ -30,9 +31,7 @@
> 
>  if [ "$TERM" = "" ]
>  then
> -       CAN_I386=`/usr/bin/isalist | grep i386`
> -       if [ -n "$CAN_I386" ]
> -       then
> +       if [ `uname -p` = "i386" ]; then
>                 TERM=sun-color
>         else
>                 TERM=sun
> bloody(~)[1]%
> 
> 
> So those of you with altered /etc/profile or /etc/.login files will
> need to be aware you'll be getting .new ones installed on the next
> bloody, or with r151014 if you don't do bloody.
> 
> FYI!
> Dan
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From danmcd at omniti.com  Fri Feb 27 23:03:41 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 18:03:41 -0500
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their boxes?
In-Reply-To: <7dd8558e38d7e3d14857227692fcf699@blackdot.be>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
	<6CCB52C9-3873-4BB6-A071-72AFB879845F@omniti.com>
	<7dd8558e38d7e3d14857227692fcf699@blackdot.be>
Message-ID: <5916276A-8F80-436D-BF4A-7F52E66D834E@omniti.com>


> On Feb 27, 2015, at 6:02 PM, Jorge Schrauwen <sjorge+ml at blackdot.be> wrote:
> 
> Add a note to the release notes when the time comes, should be good enough?

Yes, some of the myriad docs I need to write in the next month+.  Also, it's a flag day for the next bloody I push out the door.

Dan


From omnios at citrus-it.net  Fri Feb 27 23:16:33 2015
From: omnios at citrus-it.net (Andy)
Date: Fri, 27 Feb 2015 23:16:33 +0000 (GMT)
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
 their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <alpine.GSO.2.00.1502272310370.28830@areb.pvgehf-vg.arg>



On Fri, 27 Feb 2015, Dan McDonald wrote:

; This is a serious question.  A recent illumos push mildly affects people who have, but in OmniOS's case, it's a mild change (not a horrible one).
;
; I'd appreciate some on-list feedback about who does and doesn't change their /etc/profile or /etc/.login vs. what we ship.

We change /etc/profile here (and /etc/zshrc since that's our default
shell). Nothing major or that can't be put back following the update.

-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From lotheac at iki.fi  Sat Feb 28 00:59:45 2015
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Sat, 28 Feb 2015 02:59:45 +0200
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
 their boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <20150228005945.GA3762@gutsman.lotheac.fi>

On Fri, Feb 27 2015 16:27:18 -0500, Dan McDonald wrote:
> This is a serious question.  A recent illumos push mildly affects
> people who have, but in OmniOS's case, it's a mild change (not a
> horrible one).

Which one, and how?

> I'd appreciate some on-list feedback about who does and doesn't change
> their /etc/profile or /etc/.login vs. what we ship.

I would expect very nearly everyone to change them. That's why they're
in /etc, right?

> The next bloody update (and r151014) will alter the default file, and
> because of IPS, it won't be replaced IF the file was altered from the
> stock configuration.  Fortunately for OmniOS, it's mostly a
> don't-care.

Sure - as I understand it, this is how user-modifiable configuration
files are normally handled with IPS. What's the problem behind the
seriousness?

-- 
Lauri Tirkkonen | lotheac @ IRCnet

From danmcd at omniti.com  Sat Feb 28 01:02:02 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 27 Feb 2015 20:02:02 -0500
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their boxes?
In-Reply-To: <20150228005945.GA3762@gutsman.lotheac.fi>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
	<20150228005945.GA3762@gutsman.lotheac.fi>
Message-ID: <047AEDCF-6511-4FA0-8E00-6BF601730738@omniti.com>

Don't want people freaking out. If we had NOT ALREADY removed machid(1), it woiuld be a very noticeable problem.  As it is, it's more of a "oh, there's a .new version."

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Feb 27, 2015, at 7:59 PM, Lauri Tirkkonen <lotheac at iki.fi> wrote:
> 
>> On Fri, Feb 27 2015 16:27:18 -0500, Dan McDonald wrote:
>> This is a serious question.  A recent illumos push mildly affects
>> people who have, but in OmniOS's case, it's a mild change (not a
>> horrible one).
> 
> Which one, and how?
> 
>> I'd appreciate some on-list feedback about who does and doesn't change
>> their /etc/profile or /etc/.login vs. what we ship.
> 
> I would expect very nearly everyone to change them. That's why they're
> in /etc, right?
> 
>> The next bloody update (and r151014) will alter the default file, and
>> because of IPS, it won't be replaced IF the file was altered from the
>> stock configuration.  Fortunately for OmniOS, it's mostly a
>> don't-care.
> 
> Sure - as I understand it, this is how user-modifiable configuration
> files are normally handled with IPS. What's the problem behind the
> seriousness?
> 
> -- 
> Lauri Tirkkonen | lotheac @ IRCnet

From henson at acm.org  Sat Feb 28 03:58:35 2015
From: henson at acm.org (Paul B. Henson)
Date: Fri, 27 Feb 2015 19:58:35 -0800
Subject: [OmniOS-discuss] Who alters /etc/profile or /etc/.login on
	their	boxes?
In-Reply-To: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
Message-ID: <237901d0530a$d5f1b3f0$81d51bd0$@acm.org>

> From: Dan McDonald
> Sent: Friday, February 27, 2015 1:27 PM
> 
> I'd appreciate some on-list feedback about who does and doesn't change
their
> /etc/profile or /etc/.login vs. what we ship.

I will raise my hand here. I really wish upstream would've accepted the
proposal for /etc/profile.d, as that would remove my need to edit the
default system stuff <sigh>. What a bike shed that turned into 8-/.

Any thoughts on incorporating such a change into omnios? The final decision
on the illumos discuss list as I recall was that distributions should just
do it themselves if they wanted to...


From sjorge+ml at blackdot.be  Sat Feb 28 08:55:33 2015
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Sat, 28 Feb 2015 09:55:33 +0100
Subject: [OmniOS-discuss]
 =?utf-8?q?Who_alters_/etc/profile_or_/etc/=2Elog?=
 =?utf-8?q?in_on_their=09boxes=3F?=
In-Reply-To: <237901d0530a$d5f1b3f0$81d51bd0$@acm.org>
References: <F9AD985C-0B04-4BB2-8260-BC46264DFB0C@omniti.com>
	<237901d0530a$d5f1b3f0$81d51bd0$@acm.org>
Message-ID: <a96d1410d85acdc4d5e52c3b7f209607@blackdot.be>





On 2015-02-28 04:58, Paul B. Henson wrote:
>> From: Dan McDonald
>> Sent: Friday, February 27, 2015 1:27 PM
>> 
>> I'd appreciate some on-list feedback about who does and doesn't change
> their
>> /etc/profile or /etc/.login vs. what we ship.
> 
> I will raise my hand here. I really wish upstream would've accepted the
> proposal for /etc/profile.d, as that would remove my need to edit the
> default system stuff <sigh>. What a bike shed that turned into 8-/.
> 
> Any thoughts on incorporating such a change into omnios? The final 
> decision
> on the illumos discuss list as I recall was that distributions should 
> just
> do it themselves if they wanted to...
> 
+1 on this!

> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

