How long will a temporary MEMORY table persist if I don't drop it (MySQL)

Question

I'm using a recursive stored procedure in MySQL to generate a temporary table called id_list, but I must use the results of that procedure in a follow up select query, so I can't DROP the temporary table within the procedure...

BEGIN;

/* generates the temporary table of ID's */
CALL fetch_inheritance_groups('abc123',0);

/* uses the results of the stored procedure in the WHERE */
SELECT a.User_ID
FROM usr_relationships r 
INNER JOIN usr_accts a ON a.User_ID = r.User_ID 
WHERE r.Group_ID = 'abc123' OR r.Group_ID IN (SELECT * FROM id_list) 
GROUP BY r.User_ID;

COMMIT;

When calling the procedure, the first value is the top ID of the branch I want, and the second is the tier which the procedure uses during recursions. Prior to the recursive loop it checks if tier = 0 and if it is it runs:

DROP TEMPORARY TABLE IF EXISTS id_list;
CREATE TEMPORARY TABLE IF NOT EXISTS id_list (iid CHAR(32) NOT NULL) ENGINE=memory;

So my question is: If I don't DROP the temporary MEMORY table at the end of the procedure, or within my transaction, how long will that table persist in memory? Is it automatically dropped once the session ends, or will it remain in memory as long as the connection is open?

**N.B. The obvious answer might be to drop the temp table prior to the commit statement, but lets assume for a moment that I can't do that.*

---

EDIT: To be a little more precise, what if persistent connections are employed, will the table persist through multiple requests? So far it seems that it will and that we would need to explicitly remove the temp table to free up that resource.

---

UPDATE: Based on the advice from the commenters, I've found a way of adjusting my stored procedure so that I can utilize the TEMP MEMORY table, but be able to explicitly DROP it at the end...

Rather than just calling the stored procedure and using the remaining TEMP table to gather the results in the actual query, I've changed the CALL format to use a third OUT variable like so:

CALL fetch_inheritance_groups('abc123','0',@IDS);

... then within the stored procedure, I added a second IF tier = 0 at the very end with the following:

IF tier = 0
    THEN
    SELECT GROUP_CONCAT(DISTINCT iid SEPARATOR ',') FROM id_list INTO inherited_set;
    DROP TEMPORARY TABLE IF EXISTS id_list;
END IF;

So the result of the stored procedure is now a comma separated list of ID's that is compatible with FIND_IN_SET, and so the final query was modified so that:

WHERE r.Group_ID = 'abc123' OR r.Group_ID IN (SELECT * FROM id_list)

... is now ...

WHERE r.Group_ID = 'abc123' OR FIND_IN_SET(r.Group_ID,@IDS)

Voila! Thanks to the commentors for your input, and for giving me the reason I needed to try a little harder :)

Answer 1

What is funny about temporary tables in a stored procedure is not so much the transient existence of the table (which gets dropped upon the DB connection's termination), but the scope of the stored procedure.

Someone asked this question on StackOverflow : Scope of temp tables created in MySQL stored procedure. It has been over a year and nobody answered the question? Let me set the record straight. The fact is: The temp table exists inside and outside of the Stored Procedure, but you can do things with the temporary table only inside the scope of a running Stored Procedure.

According to the Book

Chapter 5 has a subheading Returning Result Sets to Another Stored Procedure.

It says in paragraph 2 on Page 117:

Unfortunately, the only way to pass a result set from one stored procedure to another is to pass the results via a temporary table. This is an awkward solution b, and -- because the temporary table has scope throughout the entire session -- it creates many of the same maintainability issues raised by the use of global variables. but if one stored program needs to supply another stored program with results, then a temporary table can be the best solution.

Looking back at the StackOverflow question, I can see someone called the Stored Procedure from the mysql client. Since the mysql client is not a Stored Procedure, the results cannot be manipulated the mysql client level via DML other than doing a SELECT to see the results. Since you calling a recursive stored procedure, you can rest assured the temp table is fully accessible for the duration of the DB Connection.

I hope this answers your question.

UPDATE 2014-01-31 11:26 EST

In your last comment, you said

If we employ persistent connections, will the MEMORY table persist through multiple REQUESTS, and it seems it will, so for performance sake, I'm assuming that using this method will *REQUIRE us to explicitly DROP the temporary MEMORY table. Do I assume correctly?

Yes and No. I say Yes because it is one way to do it. I say no because another way to do it is:

CREATE TEMPORARY TABLE IF NOT EXISTS id_list (iid CHAR(32) NOT NULL) ENGINE=memory;
TRUNCATE TABLE id_list;

Whichever way you choose, the operation is still the same since TRUNCATE TABLE drops and recreates the table. This will not harm other DB Connections since each Connection has its own id_list table.

Answer 2

In most DBMSs temporary tables survive until the end of the current connection unless otherwise specified or unless there is an explicit transaction rollback (in some systems a rollback may only affect the content of the table, leaving the object itself around to be repopulated if needed). The table will not (by default) be visible to other connections no matter how long the connection that creates it lasts.

A quick scan on Google seems to indicate this is how mySQL operates.
_{(http://www.tutorialspoint.com/mysql/mysql-temporary-tables.htm states "by default, all the temporary tables are deleted by MySQL when your database connection gets terminated")}

There are often ways to alter these behaviours though. For instance in MS SQL Server you can create a temporary table that is visible to all connections instead of just the current one by giving it a name starting ##.

I always drop temporary tables as soon as they are no longer needed to avoid possible confusion. I have been bitten before where connection pooling resulted in temporary table creation causing errors because a temporary table of the same name had been created but not destroyed in a previous action that used the current connection.

Answer 3

CREATE TEMPORARY TABLE  IF NOT EXISTS temp (Months VARCHAR(50),Sequence INT)
AS (
SELECT 
CONCAT(MONTHNAME(m1),' ',YEAR(m1)) AS Months,CONVERT(m1,DATE) AS Sequence
FROM
(
SELECT 
('2014-01-01' - INTERVAL DAYOFMONTH('2014-01-01')-1 DAY) 
+INTERVAL m MONTH AS m1
FROM
(
SELECT @rownum:=@rownum+1 AS m FROM
(SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t1,
(SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t2,
(SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t3,
(SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t4,
(SELECT @rownum:=-1) t0
) d1
) d2 
WHERE m1<= '2015-07-30'
ORDER BY m1
) ;

SELECT t.Months,A.OpenCount,A.CloseCount FROM Temp T
 LEFT JOIN ( SELECT  CONCAT(MONTHNAME(e.dtcdate),' ',YEAR(e.dtcdate)) AS Months,
 ( SELECT  COUNT(csr.ddlcsstatus) FROM csrcrn_frmempengagreqs csr 
 JOIN master_detail md ON md.masterDetailId=csr.ddlcsstatus WHERE md.abbr='open' AND csr.dtcdate >='2014-01-01' AND csr.dtcdate <='2015-07-30' AND csr.ddlArea=e.ddlArea ) AS OpenCount
 ,
 ( SELECT  COUNT(csr.ddlcsstatus) FROM csrcrn_frmempengagreqs csr 
 JOIN master_detail md ON md.masterDetailId=csr.ddlcsstatus WHERE md.abbr='Close' AND csr.dtcdate >='2014-01-01' AND csr.dtcdate <='2015-07-30' AND csr.ddlArea=e.ddlArea ) AS CloseCount

 FROM csrcrn_frmempengagreqs e 
 INNER JOIN master_detail m ON e.ddlcsstatus=m.masterDetailId 
  WHERE  e.dtcdate >='2014-01-01' AND e.dtcdate <='2015-07-30' 
 GROUP BY MONTH(e.dtcdate) ORDER BY e.dtcdate 
 ) A ON CONVERT(A.Months,CHAR(20))=CONVERT(T.Months,CHAR(20)) 
       ORDER BY T.Sequence; 
       DROP TEMPORARY TABLE  IF EXISTS temp;

/* given query give result successfully... when put this query in USP then showing error plz help me..proc is given below */

DELIMITER $$

DROP PROCEDURE IF EXISTS `usp_GetEngMonthlyChart_Test`$$

CREATE DEFINER=`root`@`%` PROCEDURE `usp_GetEngMonthlyChart_Test`(IN DateFrom DATE,IN DateTo DATE)
BEGIN
      -- SET @strWhere= CONCAT(' AND CSR.dtcInductionDate BETWEEN ''',CONVERT(DateFrom,DATE),''' AND ','''',CONVERT(DateTo,DATE),''''); 


    SET @strSql=CONCAT(' 

    CREATE TEMPORARY TABLE  IF NOT EXISTS temp (Months VARCHAR(50),Sequence INT)
    AS (
    SELECT 
    CONCAT(MONTHNAME(m1),'' '',YEAR(m1)) AS Months,CONVERT(m1,DATE) AS Sequence
    FROM
    (
    SELECT 
    (''',DateFrom,''' - INTERVAL DAYOFMONTH(''',DateFrom,''')-1 DAY) 
    +INTERVAL m MONTH AS m1
    FROM
    (
    SELECT @rownum:=@rownum+1 AS m FROM
    (SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t1,
    (SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t2,
    (SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t3,
    (SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4) t4,
    (SELECT @rownum:=-1) t0
    ) d1
    ) d2 
    WHERE m1<= ''',DateTo,'''
    ORDER BY m1
    )' );   

         SET @strSql=CONCAT(@strSql,'; GO SELECT t.Months,A.OpenCount,A.CloseCount FROM Temp T
     LEFT JOIN ( SELECT  CONCAT(MONTHNAME(e.dtcdate),'' '',YEAR(e.dtcdate)) AS Months,
     ( SELECT  COUNT(csr.ddlcsstatus) FROM csrcrn_frmempengagreqs csr 
     JOIN master_detail md ON md.masterDetailId=csr.ddlcsstatus WHERE md.abbr=''open'' AND csr.dtcdate >=''',DateFrom,
     ''' AND csr.dtcdate <=''',DateTo,''' AND csr.ddlArea=e.ddlArea ) AS OpenCount
     ,
     ( SELECT  COUNT(csr.ddlcsstatus) FROM csrcrn_frmempengagreqs csr 
     JOIN master_detail md ON md.masterDetailId=csr.ddlcsstatus WHERE md.abbr=''Close'' AND csr.dtcdate >=''',DateFrom,
     ''' AND csr.dtcdate <=''',DateTo,''' AND csr.ddlArea=e.ddlArea ) AS CloseCount

     FROM csrcrn_frmempengagreqs e 
     INNER JOIN master_detail m ON e.ddlcsstatus=m.masterDetailId 
      WHERE  e.dtcdate >=''',DateFrom,''' AND e.dtcdate <=''',DateTo,''' 
     GROUP BY MONTH(e.dtcdate) ORDER BY e.dtcdate 
     ) A ON CONVERT(A.Months,CHAR(20))=CONVERT(T.Months,CHAR(20)) 
           ORDER BY T.Sequence; 
           DROP TEMPORARY TABLE  IF EXISTS temp;'); 

    SELECT @strSql;
    PREPARE stmt FROM @strSql;
        EXECUTE stmt;
        DEALLOCATE PREPARE stmt;
    END$$

DELIMITER ;

CALL usp_GetEngMonthlyChart_Test('2014-01-01','2015-07-30')