Is the code shown under "Relevant code:" (in the question) the exact code being executed, as in copied and pasted into here and not re-typed? I ask because the file extension on the sqlserver_backup_cert file is different between the BACKUP and CREATE (i.e. restore) statements. It is backed-up as .cer but then you are looking for .crt in the CREATE.
netstat -an | grep LISTEN | grep 5444 will show if there's a process listening on that port.
netstat -an | grep LISTEN will show you all ports with a process listening on them. You can then use lsof to see which processes are actually using a given port.
There's no such concept of "find out the application port even the application was shutdown", as ...
My windows user was not allowed to run netstat -abn as suggested in geographika's answer, but I could work around it:
By using tasklist | find /i "Ssms.exe", getting following result:
Ssms.exe 14144 RDP-Tcp#15 3 233.384 K
14144 in the example above is the PID, and when tried netstat -oan | find /i "14144"