The "Cannot assign requested address" part in the error message comes from the kernel TCP stack. When encountered intermittently, this typically means that the space of available sockets is exhausted because of too much sockets in wait state (
TIME_WAIT, or less probably
The range of socket ports can be output by
cat /proc/sys/net/ipv4/ip_local_port_range. The default value on a stock Linux kernel is generally
You may check the result of
netstat -ton|grep WAIT on the client(s) and on the pgBouncer's host when the system is busy. The
-o flag will show the timeout
counters related to wait states.
If the total number of TCP sockets is close to
61000-32768=28232 then exhaustion of this range is likely your problem. Since a closed socket spends 60 seconds in
TIME_WAIT state in normal condition, if a client host connects more than 28232 times in one minute, new connections will fail with the mentioned error until ports are freed.
As a first workaround, the TCP ports range may be extended:
# echo "1025 65535" >/proc/sys/net/ipv4/ip_local_port_range
If it's not satisfactory, check the
tcp_tw_reuse flags, also tunable through
They're defined as (from
tcp_tw_recycle (Boolean; default: disabled; since Linux 2.4)
Enable fast recycling of TIME_WAIT sockets. Enabling this
option is not recommended since this causes problems when work‐
ing with NAT (Network Address Translation).
tcp_tw_reuse (Boolean; default: disabled; since Linux 2.4.19/2.6)
Allow to reuse TIME_WAIT sockets for new connections when it is
safe from protocol viewpoint. It should not be changed without
advice/request of technical experts.
Personally I had success with
tcp_tw_recycle when faced with this problem with a MySQL client app, but don't take this as a recommendation, my understanding of TCP being superficial at best.