`<div dir="ltr">Hello,<div><br></div><div>We are having a very strange issue on one of servers. The issue is that fcntl locking over NFS returns 'no locks available' immediately.</div><div><br></div><div>dtrace shows that bindresvport() returns error code 125 (EADDRINUSE):</div><div><br></div><div># dtrace -n 'fbt:rpcmod:bindresvport:return /arg1 != 0/ {stack(); printf("ret = %d", arg1);}'</div><div> 9 52692 bindresvport:return</div><div> rpcmod`connmgr_get+0x560</div><div> rpcmod`connmgr_wrapget+0x63</div><div> rpcmod`clnt_cots_kcallit+0x198</div><div> rpcmod`rpcbind_getaddr+0x245</div><div> klmmod`update_host_rpcbinding+0x4f</div><div> klmmod`nlm_host_get_rpc+0x6d</div><div> klmmod`nlm_do_lock+0x10d</div><div> klmmod`nlm4_lock_4_svc+0x2a</div><div> klmmod`nlm_dispatch+0xe6</div><div> klmmod`nlm_prog_4+0x34</div><div> rpcmod`svc_getreq+0x1c1</div><div> rpcmod`svc_run+0x146</div><div> rpcmod`svc_do_run+0x8e</div><div> nfs`nfssys+0xf1</div><div> unix`_sys_sysenter_post_swapgs+0x149</div><div>ret = 125</div><div><br></div><div>netstat shows that 501 reserved ports are in BOUND state:</div><div><br></div><div># netstat -an | grep BOUND</div><div> *.935 *.* 0 0 1049740 0 BOUND</div><div> *.801 *.* 0 0 1049740 0 BOUND</div><div> *.798 *.* 0 0 1049740 0 BOUND</div><div> *.561 *.* 0 0 1049740 0 BOUND</div><div> *.613 *.* 0 0 1049740 0 BOUND</div><div> ....</div><div># netstat -an | grep BOUND | wc -l</div><div> 501</div><div><br></div><div>Has anyone seen this similar issue? is it possible to unbind those reserved ports? Rebooting the server is our last resort.</div><div><br></div><div>Any advice would be very much appreciated.</div><div><br></div><div>Thanks,</div><div><br></div><div>-Youzhong</div><div><br></div><div> </div><div><br></div><div><br></div></div>`