# [OmniOS-discuss] [developer] How to release reserved TCP port binding in BOUND state

Marcel Telka marcel at telka.sk
Mon Nov 3 21:50:21 UTC 2014

- Previous message: [OmniOS-discuss] How to release reserved TCP port binding in BOUND state
- Next message: [OmniOS-discuss] [developer] How to release reserved TCP port binding in BOUND state
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Hi Youzhong,

    On Mon, Nov 03, 2014 at 04:14:55PM -0500, Youzhong Yang via illumos-developer wrote:
    > Hello,
    >
    > We are having a very strange issue on one of servers. The issue is that
    > fcntl locking over NFS returns 'no locks available' immediately.
    >
    > dtrace shows that bindresvport() returns error code 125 (EADDRINUSE):
    >
    > # dtrace -n 'fbt:rpcmod:bindresvport:return /arg1 != 0/ {stack();
    > printf("ret = %d", arg1);}'
    >   9  52692              bindresvport:return
    >               rpcmod`connmgr_get+0x560
    >               rpcmod`connmgr_wrapget+0x63
    >               rpcmod`clnt_cots_kcallit+0x198
    >               rpcmod`rpcbind_getaddr+0x245
    >               klmmod`update_host_rpcbinding+0x4f
    >               klmmod`nlm_host_get_rpc+0x6d
    >               klmmod`nlm_do_lock+0x10d
    >               klmmod`nlm4_lock_4_svc+0x2a
    >               klmmod`nlm_dispatch+0xe6
    >               klmmod`nlm_prog_4+0x34
    >               rpcmod`svc_getreq+0x1c1
    >               rpcmod`svc_run+0x146
    >               rpcmod`svc_do_run+0x8e
    >               nfs`nfssys+0xf1
    >               unix`_sys_sysenter_post_swapgs+0x149
    > ret = 125
    >
    > netstat shows that 501 reserved ports are in BOUND state:
    >
    > # netstat -an | grep BOUND
    >       *.935                *.*                0      0 1049740      0 BOUND
    >       *.801                *.*                0      0 1049740      0 BOUND
    >       *.798                *.*                0      0 1049740      0 BOUND
    >       *.561                *.*                0      0 1049740      0 BOUND
    >       *.613                *.*                0      0 1049740      0 BOUND
    >       ....
    > # netstat -an | grep BOUND | wc -l
    >      501
    >
    > Has anyone seen this similar issue? is it possible to unbind those reserved
    > ports? Rebooting the server is our last resort.
    >
    > Any advice would be very much appreciated.

    I faced similar issue in connmgr_get().  It is filed as #1616 and the problem
    is that the dead connection is not properly closed (there seems to be missing
    connmgr_cancelconn() call somewhere), so the client could properly reconnect.
    Unfortunately, I had no time to finish the analysis of this bug.

    HTH

    --
    +-------------------------------------------+
    | Marcel Telka   e-mail:   marcel at telka.sk  |
    |                homepage: http://telka.sk/ |
    |                jabber:   marcel at jabber.sk |
    +-------------------------------------------+

- Previous message: [OmniOS-discuss] How to release reserved TCP port binding in BOUND state
- Next message: [OmniOS-discuss] [developer] How to release reserved TCP port binding in BOUND state
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list