Timeline for Export from DB2 table to CSV with minimal client memory
Current License: CC BY-SA 4.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 15 at 6:47 | answer | added | Greg | timeline score: 0 | |
| Apr 30 at 19:29 | comment | added | mao | Try to localise the problem. Write to local file-system not NFS-mounted (temporarily to compare). Check what options exist for the NFS-mounted file system to relating to memory-usage. Next, export to a pipe, use separate (i.e. non-DB2) process to consume the pipe contents and write to the NFS-mounted-file-system. Compare behaviours? Upgrade Db2 v11.1 client (i.e. to 11.1.4.7) compare results. Upgrade to v11.5.9.0 (on client), rebinding CLI packages, to compare behaviour. | |
| Apr 30 at 4:48 | comment | added | mew | @Lennart-SlavaUkraini By client, I mean the machine where the db2 command is being run. In the example above, it would be the VM. There are three entities in the example, the VM, the DB2 server, and the NFS share where the output needs to be written. | |
| Apr 29 at 6:24 | comment | added | Lennart - Slava Ukraini | Not sure what you mean by client memory, is it the memory allocated by Db2 during the export? | |
| Apr 27 at 13:01 | comment | added | mew | Sorry if that appears rude. My perspective is that this isn't a query / table problem, but rather a problem with the way I am using db2 export. | |
| Apr 27 at 12:58 | comment | added | mew |
There is an index on the K3 column, which is used in the where clause. But I'm not sure if that is relevant. The primary goal of the question is to find out what is the best way to export large volumes of data out of DB2, regardless of the query. A generic solution which could work for SELECT * FROM XYZ WHERE XYZ.col1 = 'foo', where XYZ has no keys to order by. By the questions in the comments, do I assume that the only way db2 can export is by fetching all the rows of the result-set of the query into the client memory?
|
|
| Apr 27 at 8:30 | comment | added | Lennart - Slava Ukraini | What indexes do you have on LARGE_TABLE? What is the access plan, is statistics relevant? | |
| Apr 26 at 11:23 | history | edited | mew | CC BY-SA 4.0 |
added 12 characters in body
|
| Apr 26 at 11:22 | history | edited | mew | CC BY-SA 4.0 |
added 123 characters in body
|
| Apr 26 at 11:06 | history | edited | mew | CC BY-SA 4.0 |
deleted 96 characters in body
|
| Apr 26 at 11:05 | comment | added | mew | @RohitGupta, I have added a sample query. Please let me know if any other details can help. | |
| Apr 26 at 11:03 | history | edited | mew | CC BY-SA 4.0 |
added 200 characters in body
|
| Apr 26 at 10:08 | comment | added | Rohit Gupta | You cant expect a decent answer without details. If you are not allowed to post the sql, make one up that is similar, change names of tables and fields. You need to make it easy for someone else to help you. | |
| Apr 26 at 8:47 | history | edited | mew | CC BY-SA 4.0 |
added 4 characters in body
|
| S Apr 26 at 8:47 | review | First questions | |||
| Apr 26 at 10:08 | |||||
| S Apr 26 at 8:47 | history | asked | mew | CC BY-SA 4.0 |