Timeline for PostgreSQL importing thousands of columns as an array
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 23, 2017 at 11:33 | history | edited | CommunityBot |
replaced http://stackoverflow.com/ with https://stackoverflow.com/
|
|
| Apr 13, 2017 at 12:42 | history | edited | CommunityBot |
replaced http://dba.stackexchange.com/ with https://dba.stackexchange.com/
|
|
| Mar 11, 2017 at 18:23 | comment | added | Evan Carroll | @pietrop Even if you turned off all the durability options I can't see this being even close to the speed, and it has nothing to with regular expressions (though yes perl is faster there too). It's creating rows as an intermediary storage. That's perfectly fine and likely not an issue, but I doubt it's even close to the same speed. Perl arrays are faster than extending than PostgreSQL tables, but here we're not even doing that. We're just transforming a stream. Nothing touches the disk and gets materialized except the final copy. | |
| Mar 11, 2017 at 18:17 | comment | added | Evan Carroll | @pietrop Massively slower. | |
| Mar 11, 2017 at 13:53 | comment | added | Erwin Brandstetter | @pietrop: It's pretty fast, but Perl is the master of regular expressions. That part will be faster in Perl. OTOH, if the file isn't huge, the overhead of involving another piece of software will cost more. Also: with different kinds of code: more complications, more points of failure. And: if we need to optimize performance, we can do without the regular expression. I did not bother, because the OP did not mention performance. | |
| Mar 11, 2017 at 6:32 | comment | added | pietrop | This is the cleanest solution. It would be interesting to know if it's faster than the other solutions, but my guess is that Perl is faster on string manipulation. | |
| Mar 11, 2017 at 2:06 | history | answered | Erwin Brandstetter | CC BY-SA 3.0 |