Before upgrading AWS RDS PostgreSQL version at production, I want to replay 1-hour recorded logs at the test instance.
Unfortunately pgreplay is single-threaded application, limited by single CPU speed and its replays are failing with some weird errors after a few minutes working.
On the other side, pgreplay-go requires a different input log format:
log_line_prefix='%m|%u|%d|%c|
RDS csvlog sample:
2021-09-09 17:00:00.006 UTC,"user","database",27752,"172.30.1.2:34106",613a286d.6c68,13992,
"SELECT",2021-09-09 15:29:49 UTC,229/3866470,0,LOG,00000,
"execute <unnamed>: SELECT ""jobs"".* FROM ""jobs"" WHERE ""jobs"".""deleted_at"" IS NULL
AND ""jobs"".""user_id"" = $1","parameters: $1 = '124765'",,,,,,,,"bin/rails"
pgreplay-go log sample:
2010-12-31 10:59:57.870 UTC|postgres|postgres|4d1db7a8.4227|
LOG: execute einf"ug: INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4)
2010-12-31 10:59:57.870 UTC|postgres|postgres|4d1db7a8.4227|
DETAIL: parameters: $1 = '6', $2 = 'mit Tabulator', $3 = '2050-03-31 22:00:00+00', $4 = NULL
Should I just convert one RDS csvlog line into two pgreplay-go lines and that's it?
Log parsing logic in pgreplay-go is not trivial as you can see at https://github.com/gocardless/pgreplay-go/blob/master/pkg/pgreplay/parse.go#L220
So I'm not sure that just split of the one line into two lines will be sufficient. Should I add something else maybe?
