MySQL release memory allocated for a temp table

Question

I'm using 5.6 MySQL database to insert tens of millions records between multiple tables.

Before the insertion I send all my data into a temp memory table, for that reason I have increased both tmp_table_size and max_heap_table_size to 15GB.

It works well so far, but after I explicitly drop the temp table and after my connection is closed MySQL keeps using the memory allocated for the temp table and the only way I'm able to free that memory is to restart the service.

My application runs every few hours and I eventually run out of memory after 2-3 runs. Is there another way to fix this without restarting the service?

UPD: here's the structure of the temp table:

Temp table DDL

CREATE TEMPORARY TABLE ids_temp (
  AAID binary(16) NOT NULL,
  GroupID int(10) NOT NULL,
  INDEX temp_GroupID_idx (GroupID)
) ENGINE = MEMORY

Inserts into the temp table (using MySQLCursor.executemany())

INSERT INTO ids_temp (ID, GroupID) VALUES (%s, %s)

Drop statement

DROP TEMPORARY TABLE IF EXISTS ids_temp

Inserts to other tables

INSERT IGNORE INTO data.{GroupID} (`ID`)
SELECT T.ID
FROM ids_temp T WHERE T.GroupID = {GroupId}