The primary key is a way to distinguish one row in a single table from all other rows in that same table. It is not a way to distinguish one row in the context of its associated rows from other tables.
Sometimes a table's primary key consists of a single column. A person's user_id would be an example.
Sometimes it is made up of several columns. A location is both latitude and longitude. This is known as a compound key. Sometimes one or more of those columns may also be a foreign key. This is termed a weak entity type.
To take your example - could a single row in the Orders table be distinguished from all other rows by the Order Number alone? Typically, yes. The order number is unique across the whole system. So given order number 8765 we know that's for customer A. This makes Order a strong entity type.
How about the OrderLine table? Given a single order line number, say "1", could we unambiguously find which Order that relates to? Typically no, because order line numbers start again for each Order. OrderLine is therefore a weak entity because its primary key (order number, order line number) requires the primary key from another related table, viz. Order.
So according to the business rules it makes no sense for an Order to exist without the Customer but according to the database rules this is OK. An OrderLine cannot exist without the Order under either set of rules.