Are you familiar with "server versus client"? Your program (Workbench, PHP, magento, etc) is the "client". MySQL's program (
mysqld) is the server.
If both are on the same computer, the connection can be made via a "socket" and you use "localhost", but no "port".
If they are on separate computers, the client needs to specify where the server is (via hostname or IP address) and port number (or use the default of 3306).
Database products other than MySQL have similar connection protocols.
The typical beginner installs WAMP or LAMP. They contain Apache, MySQL/Workbench/phpmyadmin, and PHP/Perl/Python on Windows or Linux. Everything you need for building and publishing, say, web pages. In that context, everything is on the same computer. Under the hood, there are a lot of moving parts.
As far as connecting, you simply follow the "socket" or "localhost" path.
Meanwhile, the "big guys" have thousands of servers -- some with MySQL, some with web servers (Apache+PHP or whatever), some with "clients" (with WordPress or whatever) and they talk over the network (hostname+port). But you are not a "big guy" yet, so don't worry about this detail.