I am using Python with pandas to import a CSV file into a table in Postgres
import pandas as pd import psycopg2 from sqlalchemy import create_engine df = pd.read_csv('products.csv', sep=';', low_memory=False) engine = create_engine('postgresql://myuser:mypass@server/postgres') df.to_sql('new_table', con=engine, if_exists='append', index=False, chunksize=20000)
The .csv file size is ~10GB. I left the script running for 15 hours but it's nowhere near finishing. What better way can I use to push the db to the server?
I can't import the db from the server directly because the compressed file size is larger than the size allowed.