1

I have a recurring issue where a postgres process is pegging my cpu at > 100%. It's a strange one, you can see below the command is ./Ac2p20853 (surrounded by asterisks, added by me).

Running SELECT pid, datname, usename, query FROM pg_stat_activity shows nothing too interesting (to me at least). Simply a few inserts into pg_proc.

How would you go about debugging what is the issue here?

See below for output from ps, grepping postgres.

Thank you.

deploy@localhost:~$ ps auxww | grep ^postgres
postgres   590  0.0  0.3 309312 13768 ?        Ss   20:29   0:00 postgres: deploy template1 5.5.5.5(53394) idle
postgres  2264  0.0  0.3 309312 13748 ?        Ss   20:20   0:00 postgres: deploy template1 5.5.5.5(37818) idle
postgres  3079  0.0  0.3 309312 13776 ?        Ss   20:12   0:00 postgres: deploy template1 5.5.5.5(60006) idle
postgres  3809  0.8  0.0   4504  1656 ?        S    18:11   1:19 /bin/sh ./systemd
**postgres  3862  167  0.1 252512  5464 ?        Ssl  18:11 265:06 ./Ac2p20853**
postgres  5287  0.0  0.6 308388 24560 ?        S    Aug09   1:22 /usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf
postgres  5289  0.0  2.1 308516 87096 ?        Ss   Aug09   0:01 postgres: checkpointer process
postgres  5290  0.0  0.5 308388 22832 ?        Ss   Aug09   0:04 postgres: writer process
postgres  5291  0.0  0.2 308388  8676 ?        Ss   Aug09   0:03 postgres: wal writer process
postgres  5292  0.0  0.2 309196 11972 ?        Ss   Aug09   1:03 postgres: autovacuum launcher process
postgres  5293  0.0  0.1 163724  5004 ?        Ss   Aug09   2:34 postgres: stats collector process
postgres  6065  0.0  0.3 309312 13824 ?        Ss   20:04   0:00 postgres: deploy template1 5.5.5.5(51616) idle
postgres  7384  0.0  0.3 309312 13832 ?        Ss   20:40   0:00 postgres: deploy template1 5.5.5.5(55916) idle
postgres 11467  0.1  4.0 328788 161612 ?       Ss   Aug09  10:50 postgres: deploy db_production [local] idle
postgres 11638  0.0  4.3 339336 174368 ?       Ss   Aug09   4:52 postgres: deploy db_production [local] idle
postgres 11812  0.0  0.5 316092 21496 ?        Ss   Aug09   0:00 postgres: deploy postgres [local] idle
postgres 12707  0.0  0.4 315660 16820 ?        Ss   Aug09   0:00 postgres: deploy db_production [local] idle
postgres 17511  0.0  0.3 309312 13716 ?        Ss   20:33   0:00 postgres: deploy template1 5.5.5.5(33460) idle
postgres 17968  0.0  0.3 309312 13804 ?        Ss   20:44   0:00 postgres: deploy template1 5.5.5.5(34548) idle
postgres 18974  0.0  0.3 309312 13768 ?        Ss   20:16   0:00 postgres: deploy template1 5.5.5.5(52290) idle
postgres 19017  0.0  0.3 309312 13752 ?        Ss   20:24   0:00 postgres: deploy template1 5.5.5.5(37348) idle
postgres 21120  0.0  0.3 309312 13788 ?        Ss   20:08   0:00 postgres: deploy template1 5.5.5.5(33084) idle
postgres 24189  0.0  0.3 309312 13744 ?        Ss   20:36   0:00 postgres: deploy template1 5.5.5.5(35660) idle
postgres 31498  0.0  0.3 309312 13804 ?        Ss   20:48   0:00 postgres: deploy template1 5.5.5.5(39618) idle
2

Most likely you've been hacked and are now mining cryptocurrency for the hacker.

Chances are, you have a weak password for your postgres superuser account (probably named "postgres") and you let anyone in the world connect and attempt to log in, which they do until they guess your weak password. Or maybe you have no password at all.

  • Thanks, that does seem likely. However, still stumped on the vector. /etc/shadow shows * for postgres user password so the account should be 'locked' and not able to be logged in. – 46and2 Aug 14 '18 at 21:44
  • They log in to the PostgreSQL server using an account in the PostgreSQL database, not the OS user account named 'postgres'. So you would be looking in pg_authid or pg_shadow database tables, not /etc/shadow. But once you have super-user access to the database, you can leverage that to get shell access as the OS user 'postgres' (or whoever the database runs as) without needing to log in to the OS user explicitly. – jjanes Aug 15 '18 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.