0

I'm trying to copy data from S3 to PostgreSQL (Amazon AWS RDS). I've set up the following:

CREATE EXTENSION aws_s3 cascade schema reporting;
CREATE DATABASE my_database;
\connect my_database;  -- reconnect to my_database
CREATE SCHEMA my_schema;
SET password_encryption = 'md5'; -- required by the Airflow connector I'm using
CREATE USER airflow_devel WITH PASSWORD 'my_password';
ALTER DATABASE my_database OWNER airflow_devel;
ALTER USER airflow_devel IN DATABASE my_database SET search_path TO 'my_schema';

Then I run

SELECT aws_s3.table_import_from_s3(
    '"table_name"',  /*table_name*/
    '',  /*empty column_list for all columns*/
     '(FORMAT CSV, HEADER FALSE, DELIMITER ''|'')',  /*options*/
    aws_commons.create_s3_uri(
        'my_bucket',
        'my/s3/key',
        'eu-central-1'
    )  /*s3_info*/
);

It runs fine using the master user. However, when I run from just-created user airflow_devel, I get

permission denied for schema aws_s3

I haven't found an explanation on the internet why this function and extension should not be possible from non-root users. But is this assumption wrong? What does the message permission denied for schema aws_s3 mean? I mean, aws_s3 is not a schema anyway. I'm grateful for any advice or alternatives!

3
  • aws_s3 certainly is a schema. It might also be the name of something else, too. But that doesn't mean it isn't also a schema.
    – jjanes
    Commented May 2, 2023 at 19:23
  • 1
    You created the extension in some database other than "my_database".
    – jjanes
    Commented May 2, 2023 at 19:25
  • Thanks for replying. In which DB the extension was created does not seem to matter so far. Are extensions generally stored as schemas? I can get one step further with grant usage on schema aws_s3 to airflow_devel, but then I get permission denied for function table_import_from_s3. Commented May 3, 2023 at 9:06

1 Answer 1

0

With some research I found the solution was already given here: https://stackoverflow.com/questions/66535534/permission-denied-for-function-table-import-from-s3

We need to run

GRANT USAGE ON schema aws_s3 TO airflow_devel;

GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA aws_s3 TO airflow_devel;

That makes it work for me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.