13

I'm generating a list of SQL commands to export some data that I ultimately run using psql -f. The queries all get the same subset of data, so I thought I'd factor the qualifications out and put a list of eligible user ids in a temporary tables like so

create temporary table tmp_export_users as (select id from users where ...)

then refer back to that in my \copy commands like

\copy (select ... from table where user_id in (select id from tmp_export_users)) TO 'filename.csv' WITH CSV HEADER

Those are all in the same file, one per line, and running them -f I get the error that the copy commands can't see the temporary table, so I'm guessing that the client copy command must not actually use the same postgres session as psql.

Is that correct? Is there a way to change that behavior?

1 Answer 1

20

\copy can use a temporary table.

First I tested and confirmed this with version 9.0 at the command line.
Then I created a file with SQL and psql meta command \copy using multiple temporary tables. That worked for me, too.

CREATE TEMP TABLE tmp as SELECT * FROM tbl;
\copy (SELECT * FROM tmp JOIN tbl USING (id)) TO '/var/lib/postgres/test1.csv';

Call:

psql -p5432 mydb -f test.sql

Note the terminating semicolon, which is optional at the end of a file (terminated implicitly), but required after any other SQL statement and also after the last one if executed in psql interactively.

Normally, psql meta-commands cannot be mixed with SQL on the same line in a file executed per psql -f. I quote the manual on psql:

Parsing for arguments stops at the end of the line, or when another unquoted backslash is found. An unquoted backslash is taken as the beginning of a new meta-command. The special sequence \\ (two backslashes) marks the end of arguments and continues parsing SQL commands, if any. That way SQL and psql commands can be freely mixed on a line. But in any case, the arguments of a meta-command cannot continue beyond the end of the line.

Different rules apply after \copy, though. Essentially, psql switches back to SQL mode automatically after \copy See:

But you wrote you had all commands on separate lines. So that cannot be the explanation in your case.


All that aside, have you considered using COPY (the SQL command) instead of \copy (the psql meta-command)?

Of course, the target file would have to be local to the server not the client in this case. And different file privileges apply. The manual:

Files named in a COPY command are read or written directly by the server, not by the client application. Therefore, they must reside on or be accessible to the database server machine, not the client. They must be accessible to and readable or writable by the PostgreSQL user (the user ID the server runs as), not the client.

4
  • copy runs as the postgres user, \copy wraps copy to write to std out and get redirected to the file you send it to. You can also call psql, use \o to send output to a file, and then run a copy to stdout to get similar effect. Nov 8, 2011 at 18:23
  • To be sure I ran the test in my answer with a superuser (postgres) and a dummy user. Both work for me. Same results on v8.4. Nov 9, 2011 at 1:58
  • 1
    Yeah, whether or not the postgres unix user can access things like /tmp depends on things like whether or not SELinux is installed and if it lets it out of its box. The \copy or copy to stdout are definitely the two more reliable ways to use copy. Nov 9, 2011 at 3:36
  • 1
    Thanks for the answers everyone. Looks like I neglected to end the line that created a temporary table with a semicolon, so it didn't get created. Working as expected now
    – jkebinger
    Nov 9, 2011 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.