Microsoft has a [KB article](http://support.microsoft.com/kb/969962) about this behavior. You got a 32-bit OS with 32-bit application, so the process' address space is 4 GiB. AWE can be used to access more memory, but it is limited only for buffer pool usage. As per [the documentation](http://technet.microsoft.com/en-us/library/ms175581(v=sql.90).aspx):

> The SQL Server buffer pool can fully utilize AWE mapped memory;
> however, only database pages can be dynamically mapped to and unmapped
> from SQL Server's virtual address space and take full advantage of
> memory allocated through AWE. AWE does not directly help supporting
> additional users, threads, databases, queries, and other objects that
> permanently reside in the virtual address space.

Short-term solution: review that your CLR code minimizes memory usage.

For a long-term solution, upgrade to 64-bit.