As part of a different issue I had to give my login the sysadmin role. I haven't seen the problem I described above since then.
Yes, it may work now, but I am not happy with this solution, partially because I don't quite understand what was going on.
Later edit: As pointed out by Thomas in a comment, here's what's going on:
If the database exists, RESTORE permissions default to members of the sysadmin and dbcreator fixed server roles and the owner (dbo) of the database (for the FROM DATABASE_SNAPSHOT option, the database always exists).