I wish to import a 5 gigabyte text file with pgloader.
The individual records are paragraphs of html. The records include carriage returns and white space. So this not an import where it is one record per line.
Reading the pgloader howto I get the impression I can import these records by using setting lines terminated by
to a special character to demark the end of a record. The options is described as:
lines terminated by
Takes a single character as argument, which must be found inside single quotes, and might be given as the printable character itself, the special value \t to denote a tabulation character, or 0x then an hexadecimal value read as the ASCII code for the character.
This character is used to recognize end-of-line condition when reading the CSV data.
So, am I right? Can I can import these records by using setting lines terminated by
to a special character?
UPDATE: I had to get the project done, so I used some perl commands to modify the data to be one record per line. It's an undesired option because I wanted a multi-line text field.
But I still would like a way to do this. Something that allows a custom record-delimiter would probably be the answer.