The message you shared relates to failed allocation to the chunk cache, which works as a Cassandra-native page cache to store SSTable chunks in memory.
While it is of no immediate concern, it indirectly relates to read latencies, but not to write latencies - the goal of the chunk cache is to speed up reads by allowing Cassandra to read chunks of data from memory off-heap rather from disk, which is generally faster - so each time you see this message it's a missed opportunity to speed up a read request by reading data from memory.
If you see the message you shared often, it means that the node can't continuously free up space in the chunk cache at the same rate that read requests demand new data to be allocated in it. In other words, increasing the chunk cache size as you considered may help, but only to the extent that you size it equal or greater than the volume of hot data being read beyond the reallocation capabilities of the nodes, which may be tricky if the clients cause intense read periods. This requires some trial and error to get it right.
If write latencies are also affected, I would look somewhere else. For high latencies in writes, the first places I'd look would be into garbage collection activity, iowait in the commit log drive, and CPU utilization, but such issues can come from multiple different sources.