Alter ownership of a table in a schema does not work

Question

create role schema_owner;
create role table_owner; 

Current signed in role is postgres

create schema test;
grant schema_owner to postgres;
grant table_owner to postgres;
alter schema test owner to schema_owner;

This Works....

create table test.t1 (a int);
alter table test.t1 owner to table_owner;

This last statement doesn't work. Why ?

Error is : permission denied for schema test

Why the postgres user cant change the owner of table t1 in schema test to table_owner. Postgres is a already a member of table_owner and schema_owner? How can I achieve this?