Before upgrading AWS RDS PostgreSQL version at production, I want to replay 1-hour recorded logs at the test instance.

Unfortunately pgreplay is single-threaded application, limited by single CPU speed and its replays are failing with some weird errors after a few minutes working.

On the other side, pgreplay-go requires a different input log format:


RDS csvlog sample:

2021-09-09 17:00:00.006 UTC,"user","database",27752,"",613a286d.6c68,13992,
"SELECT",2021-09-09 15:29:49 UTC,229/3866470,0,LOG,00000,
"execute <unnamed>: SELECT ""jobs"".* FROM ""jobs"" WHERE ""jobs"".""deleted_at"" IS NULL
AND ""jobs"".""user_id"" = $1","parameters: $1 = '124765'",,,,,,,,"bin/rails"

pgreplay-go log sample:

2010-12-31 10:59:57.870 UTC|postgres|postgres|4d1db7a8.4227|
LOG:  execute einf"ug: INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4)
2010-12-31 10:59:57.870 UTC|postgres|postgres|4d1db7a8.4227|
DETAIL:  parameters: $1 = '6', $2 = 'mit    Tabulator', $3 = '2050-03-31 22:00:00+00', $4 = NULL

Should I just convert one RDS csvlog line into two pgreplay-go lines and that's it?

Log parsing logic in pgreplay-go is not trivial as you can see at https://github.com/gocardless/pgreplay-go/blob/master/pkg/pgreplay/parse.go#L220

So I'm not sure that just split of the one line into two lines will be sufficient. Should I add something else maybe?

1 Answer 1


Because changing log_line_prefix on RDS is not supported, I wrote a dead-simple log converter from RDS csvlog to pgreplay-go log, works for SELECT only load


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.