Invoke-Sqlcmd does not support the ability to connect to a read-only secondary replica.
The SQL Server, and some of the PowerShell community, however started working with the Program Manager at Microsoft to get
SQLPS and SSMS improvements organized. A Trello board was opened in the past month that we are showing by votes what priority we want Microsoft to take on the improvements. This has actually already gotten some things added to the SSMS March preview.
I created a card for the Application Intent functionality so go vote it up and the connect item linked.
Right now your only option is to call
sqlcmd within PowerShell using the