Essentially, yes. An Azure Database for MySql is similar to an Azure SQL Database Elastic Pool or Azure SQL Database Managed Instance.
Within an Azure Database for MySQL server, you can create one or
multiple databases. You can opt to create a single database per server
to use all the resources or to create multiple databases to share the
resources. The pricing is structured per-server, based on the
configuration of pricing tier, vCores, and storage (GB). For more
information, see Pricing tiers.
An Azure SQL Database Elastic Pool is similarly:
Azure SQL Database elastic pools are a simple, cost-effective solution
for managing and scaling multiple databases that have varying and
unpredictable usage demands. The databases in an elastic pool are on a
single server and share a set number of resources at a set price.
Elastic pools in Azure SQL Database enable SaaS developers to optimize
the price performance for a group of databases within a prescribed
budget while delivering performance elasticity for each database.
Only Azure SQL Database has the feature to have lots of databases with separate physical resources in the same logical server. With Azure Database for MySql if you want two databases to have their own dedicate resources, you need two seperate Azure Database for MySQL servers.