0

I am trying to emulate MERGE behavior with pl/pgsql:

-- Generate the data from funtion
CREATE TEMP TABLE temp_x  (id int, id2 int, data text, created_at timestamp, updated_at timestamp)  ON COMMIT DROP;
INSERT INTO temp_x SELECT * FROM set_gernating_function(p);

-- DELETE record with same id2
DELETE FROM x WHERE NOT EXISTS (SELECT 1 FROM temp_x WHERE temp_x.id=x.id) AND id2=p.id2;

-- UPSERT by (id, id2)
INSERT INTO x
    SELECT * FROM temp_x z
  ON CONFLICT(id, id2) DO UPDATE
    SET
      updated_at=excluded.updated_at,
      data=excluded.data;

But the temp table usage is generating bloat in pg_class and pg_attr table very fast -- Faster then I could do vacuum -- and affect other queries. Any idea?

There are some restrictions:

  1. Preserve the created_at time if the id already exist
  2. x have a delete-trigger, so no unnecessary delete.
  3. The set_generating_function is slow
  4. The size for temp_x is small. (< 50)
  5. The run rate is very high (thousands per second)
4
  • The delete statement in your question seems to be incomplete. The condition AND id2=p.id refers to an alias p that is not present. Should that be part of the co-related sub-query?
    – user1822
    Commented Feb 25, 2016 at 10:16
  • Can't the temp table have a session lifetime instead of a transaction lifetime? Or are you doing this process only once per session? Commented Feb 25, 2016 at 15:02
  • @a_horse_with_no_name the alias p is a custom type from function parameter. Should be irrelevance here.
    – J-16 SDiZ
    Commented Feb 26, 2016 at 4:14
  • @DanielVérité I have doing ~10 process per transaction; millions per session. Per transaction temp table does not help much. Per session temp table make the temp table bloat.
    – J-16 SDiZ
    Commented Feb 26, 2016 at 4:16

1 Answer 1

3

You don't need the temp table:

You can do this in a single statement:

with temp_x (id, id2, data, created_at, updated_at) AS (
   SELECT * 
   FROM set_gernating_function(p)
), deleted as (
  DELETE FROM x 
  WHERE NOT EXISTS (SELECT 1 
                    FROM temp_x 
                    WHERE temp_x.id = x.id) 
    AND id2=p.id2
)
-- UPSERT by (id, id2)
INSERT INTO x
SELECT * 
FROM temp_x z
  ON CONFLICT(id, id2) DO UPDATE
    SET updated_at = excluded.updated_at,
        data = excluded.data;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.