the rds postgres instance has a parameter group. Check in rds what parameter group is defined : RDS -> Databases -> choose your db -> Configuration -> parameter group. If it's the default one , create a new one and change the track_activity_query_size to the value you need. Apply the new parameter group or change the actual one. The parameter is static so a restart is needed in any case.
Check this link please : https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html