0

Problem description

This week we had an issue in production, where an SQL script that was executed ended up with a different result than what we had tested before locally, in Docker and on our integration and pre-production stages. Our SQL scripts are executed using Flyway.

The script does the following:

  • add two new columns to an existing table.
  • reorder the table's columns, so the two new columns are at position 3 and 4 (Yes, I do know that the order of columns should not matter in a relational database, but it is a request from the customer).

The script we created did work without errors and resulted in the exected table definition on all environments, except in production.

Code

The table definition before the script run was something like this:

create table ABC (
    id int primary key,
    a int,
    b int,
    c int,
    d int,
    e int,
    f int
);

The migration script was something like this:

alter table ABC add x int;
alter table ABC add y int;

begin
  sp_set_column_position('ABC', 'X', 3);
  sp_set_column_position('ABC', 'Y', 4);
end;
/

This is my utility procedure sp_set_column_position, which uses the "feature" that columns which are set from invisible to visible are placed at the end of the table to

CREATE OR REPLACE PROCEDURE sp_set_column_position(in_table IN VARCHAR2, in_column IN VARCHAR2, in_position IN INT)
AS
  columns_to_modify clob;
  columns_invisible clob;
  columns_visible clob;
BEGIN
  -- set column which should be positioned to "invisible"
  -- this means the column_id will be null and the column to be positioned will not appear in the following select below
  execute immediate 'alter table "' || in_table || '" modify ("' || in_column || '" invisible)';

  -- collect all the column names starting from the new position to the end of the table
  select listagg(col.column_name, ',') within group (order by col.column_id) into columns_to_modify
  from user_tab_cols col
  where col.table_name = in_table
    and col.column_id >= in_position;

  columns_invisible := '"' || replace(columns_to_modify, ',', '" invisible, "') || '" invisible';
  columns_visible := '"' || replace(columns_to_modify, ',', '" visible, "') || '" visible';

  -- set all the collected columns to "invisible"
  execute immediate 'alter table "' || in_table || '" modify (' || columns_invisible || ')';

  -- set the column to be positioned to "visible"
  execute immediate 'alter table "' || in_table || '" modify ("' || in_column || '" visible)';

  -- set the collected columns to "visible" again
  execute immediate 'alter table "' || in_table || '" modify (' || columns_visible || ')';
END;
/

After the script was run in production it resulted in the following column order:

id, a, x, y, b, c, e, f, d

The script was run using Flyway and it did complete without any errors.

Question

  1. Is there something wrong with my procedure?
  2. What could be the reason that this works locally, in Docker and even our integration and pre-production stages, but not in production?
3
  • 1
    Unable to reproduce, column order is correct. Maybe the original column order was id,a,b,c,e,f,d. Commented Jul 1, 2022 at 15:40
  • This was my first guess as well. But we reverted the database to a snapshot which was taken before the script was run and the order was correct before. Also we tried the same script again to rule out weird runtime behaviors, but the result was the same. Also we manually executed the select statement from my sp_set_column_position procedure in production. This also resulted in the expected column list and order.
    – Dave
    Commented Jul 1, 2022 at 16:11
  • Why do you expect alter table to modify columns in any deterministic order? You shouldn't really rely on a side effect of making columns invisible and visible. The only reliable approach would be to re-create the table with the desired order of columns. If your customer want a silly thing, they should pay the price (and so should you, for not convincing them otherwise).
    – mustaccio
    Commented Jul 1, 2022 at 17:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.