thousands of ips on one machine

Discussion in 'Noob Central' started by nickphx, Sep 30, 2011.

  1. nickphx

    nickphx VIP

    Joined:
    Apr 2, 2011
    Messages:
    1,139
    Likes Received:
    363
    Trophy Points:
    83
    Gender:
    Male
    Location:
    guadalajara, chiuhuahua
    Maybe this is common knowledge, it took me a couple hours to figure out.. I was having a problem across several machines where opening new connections would incur a significant delay (500-750ms).. The problem would only happen after I configured over 10k ips.

    There's an obscure sysctl for linux that will speed up your software's bind() call that is made while creating a new connection.

    net.ipv4.ip_nonlocal_bind

    This tells the kernel to allow ip addresses that do not exist on the interface to be used. In the kernel's net code, setting this sysctl to 1 tells the netcode to not check the interfaces to see if the ip address exists. Bypassing this check when you have 10's of thousands of ips on a server makes a significant difference in opening new connections. The only downside is you will not receive an error while attempting to bind a non-configured address, your connection will timeout..

    Code:
    err = -EADDRNOTAVAIL;
    	if (!sysctl_ip_nonlocal_bind &&
    	    !(inet->freebind || inet->transparent) &&
    	    addr->sin_addr.s_addr != htonl(INADDR_ANY) &&
    	    chk_addr_ret != RTN_LOCAL &&
    	    chk_addr_ret != RTN_MULTICAST &&
    	    chk_addr_ret != RTN_BROADCAST)
    		goto out;
    
     

Share This Page