I am importing a large number of large files into a number of tables to be partitioned using loops within an anonymous plpgsql code block
$do$ BEGIN FOR yyyy in 2012..2016 THEN EXECUTE $$COPY table$$||yyyy||$$ FROM 'E:\data\file$$||yyyy||$$.csv DELIMITER ',' CSV;$$; END LOOP; END; $do$ LANGUAGE plpgsql
This entire process should take about 15 hours and I'm hoping that all the imports won't be rolled back if there's an import error at some point.
COMMIT doesn't work within stored functions bc the entire function is treated as a single transaction.
From the documentation for
The code block is treated as though it were the body of a function with no parameters, returning void. It is parsed and executed a single time.
I'm assuming this means that the entire
$do$ is one transaction, and so commits within the block won't work. Am I correct?