Can we route read-only connection to readable secondary replica without setting connection string (ApplicationIntent = ReadOnly)?
No, you cannot route read-only connection request to readable-secondary without ApplicationIntent=ReadOnly connection parameter.
However, you can connect to readable secondary directly without using listener, but as it is mentioned on the other answer, in case the secondary replica is down, HA is not achievable, in this case.
I could not think of any such specific use case. I hope it helps!
For configuring read-only routing, check this page. Thanks!