Debugging Postgres error: "could not send data to client: Broken pipe"

Question

I'm running a Django 1.8 site, served by nginx and Gunicorn, with a Postgres 9.4 backend.

When I make lots of data-intensive calls to the site's API in succession, I've noticed about 1 in 100 queries failing, and serving up nginx 404 pages.

My first assumption was that the Postgres database was creaking under the load, but the Postgres logs in fact show me this:

2015-08-06 14:33:43 UTC:127.0.0.1(42287):dbuser@dbname:[3737]: LOG:  could not send data to client: Broken pipe
2015-08-06 14:33:43 UTC:127.0.0.1(42287):dbuser@dbname:[3737]: STATEMENT:  SELECT ... (psql statement)
2015-08-06 14:33:43 UTC:127.0.0.1(42287):dbuser@dbname:[3737]: FATAL:  connection to client lost
2015-08-06 14:33:43 UTC:127.0.0.1(42287):dbuser@dbname:[3737]: STATEMENT:  SELECT ... (psql statement)

I think what this is saying is that Postgres is actually running the queries OK, but that it can't send data through to the client.

So I think to debug this problem, I should actually look at Django/Gunicorn/nginx instead.

Have I understood this correctly?

Answer 1

Yes, it seems like you've understood this correctly. The client sent the data to a postgresql backend, and then abruptly disconnected (or was killed). The message is basically "I have an answer for you, but I have nowhere to send it!"

I would definitely look at the logs of the other services and compare timestamps to try and look deeper into what is going on.

Hope that helps. =)