I would try to answer the questions point wise
After few days if i check in task manager, it shows the load of 31 GB Memory but actually maximum is alloted to SQL is 28 GB and nothing is running on he server
First please note that task manager is not correct place to look for SQL Server memory utilization especially when SQL Server service account has locked pages in memory privilege. It will not show correct value and will only show process private bytes not the*lockecD* memory. Since you have SQL server 2008 R2 please use below query to find memory utilized by SQL Server
from sys. dm_os_process_memory
Now as to why it is taking memory more than 28G because 28 G is restriction on BUFFER POOL not on total memory that can be consumed by SQL Server. Buffer pool only caters to request for memory which requires <= 8 KB pages. For any memory requirement which requires continuous memory greater than 8 KB windows API directly allocates them NOT FROM THE BUFFER POOL. So this outside allocation causes SQL Server memory consumption to increase.
Following features are given memory outside from buffer pool IN SQL SERVER 2008 R2 the list has been changed in SQL Server 2012
Extended stored procs
Linked Server memory
Thread Stack memory
Memory for Database mail
I am sure you must be using some of these features
Why its increasing to the maximum limit? And where this 31 GB used? If its used in cache then how to clear the cache without restarting the SQL?
Its increasing to maximum limit because this is how SQL Server is designed to use memory. SQ Server will take as much memory as possible to cache data pages and other objects as well so that when request comes for reading a page it wont have to generate a request for physical I/O and then get the page from disk to memory the whole process is costly and depending on the load on Server can take time. On the contrary if page is avaialble in memory your request would be satisfied immediately. But this is not a drawback it would trim down its memory consumption when SQLOS asks it to do so because it was requested by windows OS that it would require more memory so this whole process is dynamic. So NO NEED to worry whem SQL Server is utilizing memory.
For more details about memory please read below article
PS: Do revert if you still need clarification