To disable TDE you must first remove the candidate database from the Availability Group - the group itself however does not need to be dropped.
To do this, the following code is needed:
ALTER DATABASE [YourDatabase] SET HADR OFF;
Once this is complete, you may then disable TDE on the PRIMARY replica:
ALTER DATABASE [YourDatabase] SET ENCRYPTION OFF;
The process of decryption is asynchronous, and can be monitored by looking at the percent_complete column in the sys.dm_database_encryption_keys DMV
One thing to note, is that although the database itself is encrypted, if you aren't careful you can end up with part of the log file still encrypted - this is only a problem if you will be dropping the encryption certificates from the offending servers, or trying to take a backup and restore this database to a server that doesn't hold this encryption certificate.