Note you say this is in the server log and there are no known problems with clients. Consequently, I don't think that it is libpq per se. Additionally on doing some additional searches, it looks like this is a standard Windows "connection refused" message, which suggests that PostgreSQL is trying to connect to something and being refused.
If you are getting no error messages on the clients, I think this can be a relatively low priority to solve, but I would wonder about concerns regarding performance hits down the road (if you get a connection that times out and it causes clients to hang). Consequently I do think it is worth trying to resolve if only because it takes aim at a problem hiding in the shadows that could under the right circumstances cause some grief.
A few things to check before you go further might include:
What sort of authentication are you using? SSPI? Is it possible the KDC is not up when these errors occur? Are there any stored procedures (say, written in untrusted languages like pl/perlu) which make network connections to other hosts> Do these errors happen frequently or sporadically around certain times?
The key questions to be asked now are basically "what system is this connecting to and why?" From there you should be able to resolve the issue easily enough.