Problem description
This week we had an issue in production, where an SQL script that was executed ended up with a different result than what we had tested before locally, in Docker and on our integration and pre-production stages. Our SQL scripts are executed using Flyway.
The script does the following:
- add two new columns to an existing table.
- reorder the table's columns, so the two new columns are at position 3 and 4 (Yes, I do know that the order of columns should not matter in a relational database, but it is a request from the customer).
The script we created did work without errors and resulted in the exected table definition on all environments, except in production.
Code
The table definition before the script run was something like this:
create table ABC (
id int primary key,
a int,
b int,
c int,
d int,
e int,
f int
);
The migration script was something like this:
alter table ABC add x int;
alter table ABC add y int;
begin
sp_set_column_position('ABC', 'X', 3);
sp_set_column_position('ABC', 'Y', 4);
end;
/
This is my utility procedure sp_set_column_position, which uses the "feature" that columns which are set from invisible to visible are placed at the end of the table to
CREATE OR REPLACE PROCEDURE sp_set_column_position(in_table IN VARCHAR2, in_column IN VARCHAR2, in_position IN INT)
AS
columns_to_modify clob;
columns_invisible clob;
columns_visible clob;
BEGIN
-- set column which should be positioned to "invisible"
-- this means the column_id will be null and the column to be positioned will not appear in the following select below
execute immediate 'alter table "' || in_table || '" modify ("' || in_column || '" invisible)';
-- collect all the column names starting from the new position to the end of the table
select listagg(col.column_name, ',') within group (order by col.column_id) into columns_to_modify
from user_tab_cols col
where col.table_name = in_table
and col.column_id >= in_position;
columns_invisible := '"' || replace(columns_to_modify, ',', '" invisible, "') || '" invisible';
columns_visible := '"' || replace(columns_to_modify, ',', '" visible, "') || '" visible';
-- set all the collected columns to "invisible"
execute immediate 'alter table "' || in_table || '" modify (' || columns_invisible || ')';
-- set the column to be positioned to "visible"
execute immediate 'alter table "' || in_table || '" modify ("' || in_column || '" visible)';
-- set the collected columns to "visible" again
execute immediate 'alter table "' || in_table || '" modify (' || columns_visible || ')';
END;
/
After the script was run in production it resulted in the following column order:
id, a, x, y, b, c, e, f, d
The script was run using Flyway and it did complete without any errors.
Question
- Is there something wrong with my procedure?
- What could be the reason that this works locally, in Docker and even our integration and pre-production stages, but not in production?
id,a,b,c,e,f,d
.alter table
to modify columns in any deterministic order? You shouldn't really rely on a side effect of making columns invisible and visible. The only reliable approach would be to re-create the table with the desired order of columns. If your customer want a silly thing, they should pay the price (and so should you, for not convincing them otherwise).