Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It only takes a minute to sign up.
Sign up to join this community
When copying a large amount of data (around 9GB) from CSV file into table, I get following error:
ERROR: out of memory
DETAIL: Cannot enlarge string buffer containing 1073741632 bytes by 349 more bytes.
CONTEXT: COPY column_name line 13275136
A server (postgresql 10) has 8GB of memory and database has shared_buffers set to 2GB.
Can somebody suggest a solution, please. Or problem is instead in column_name length ( according to CONTEXT: COPY column_name line 13275136 message)
DETAIL: Cannot enlarge string buffer containing 1073741632 bytes by 349 more bytes. CONTEXT: COPY column_name line 13275136
According to the error message, the offending row at this line number in the file is too large to be imported. COPY uses an internal buffer per row that can only grow to 1GB. It's impossible to import or export larger rows with COPY in existing versions of PostgreSQL (up to and including version 12)
