This is because
VIEW [ANY] DEFINITION doesn't have anything to do with retrieving results that are exposed to
public by default; this has to do with viewing the definition of objects, for example this should no longer be allowed:
DENY SELECT ON sys.databases TO public;
Or, more broadly:
DENY SELECT ON SCHEMA::sys TO public;
Note that this may cause other issues, for example I am not sure what will happen if the user tries to connect to SQL Server using Management Studio - Object Explorer probably will not load completely; IntelliSense and other features may stop working as well, since they rely on metadata access.