You said in your question

It makes me think that when you insert/update/replace the table locks itself automatically and entirely so you will not be able to select anything while the write is being done. Is this correct?

You are correct.

The MEMORY storage engine performs a full table lock for every DML statement. That is why the link you mentioned shows Lock granularity : Table just like MyISAM. You cannot disable this.

In addition to this, please keep in mind that the MEMORY storage engine still imposes some mild disk I/O for both reads and writes. Why ? An open file handle is maintained on a MEMORY table to at least check on the table's existence and accessibility (See my old most I am using the MEMORY storage engine but MySQL still writes to my disk...Why?)

Please see my other old post Is it feasible to have MySQL in-memory storage engine utilize 512 GB of RAM? on the pros and cons of using MEMORY tables.

You may have to resort to using InnoDB for such session data.

You said in your question

It makes me think that when you insert/update/replace the table locks itself automatically and entirely so you will not be able to select anything while the write is being done. Is this correct?

You are correct.

The MEMORY storage engine performs a full table lock for every DML statement. That is why the link you mentioned shows Lock granularity : Table just like MyISAM. You cannot disable this.

In addition to this, please keep in mind that the MEMORY storage engine still imposes some mild disk I/O for both reads and writes. Why ? An open file handle is maintained on a MEMORY table to at least check on the table's existence and accessibility (See my old most I am using the MEMORY storage engine but MySQL still writes to my disk...Why?)

Please see my other old post Is it feasible to have MySQL in-memory storage engine utilize 512 GB of RAM? on the pros and cons of using MEMORY tables.

You may have to resort to using InnoDB for such session data.