Yes, you can create Postgres extensions in AWS Aurora Postgres. Yes, the extension
uuid-ossp can be used in Aurora.
To get supported extension list in your AWS Aurora Postgres version, run this query:
select * from pg_available_extensions;
And also check out PostgreSQL Extensions and Modules Supported on Amazon RDS in the Amazon Relational Database Service User Guide for the list of extensions supported in AWS Aurora Postgres based on Postgres versions.
There is no difference in the extension usage for both community Postgres and Aurora Postgres if extension installed on the server. For
uuid-ossp usage, please refer to F.44. uuid-ossp in the PostgreSQL documentation.