Nothing is free. Sometime not having something isn't free either. Both having and not having declared foreign keys come with costs and benefits.
The point of a foreign key (FK) is to ensure that this column over here can only ever have values that come from that column over there1. This way we can be sure we only ever capture orders for customers that actually exist, for products we actually produce and sell. A lot of people think this is a good idea.
The reason we declare them inside the DBMS is so it can take care of enforcing them. It will never ever allow in any data that breaks the rules. Also it will never allow you to get rid of data required to enforce the rules. By delegating this task to the machine we can have confidence in the integrity of the data, no matter what its source or when it was written or which application it came through. Of course this comes with a cost. The DBMS has to check the rules are being followed, for each and every row, for each and every query, all the time. This takes time and effort, which is a load on the server. It also requires the humans to submit DML in a sequence that respects the rules. No more slyly forcing in an Order, then catching up with the admin afterwards. Oh no naughty human, you must first create the Customer, then the Product (and all the prerequisite rows) and only then may you create an Order.
Without foreign keys we are much freer with what we can do, and the order in which we can do it. Problematical rows can be removed ad hoc to allow critical processes to complete. The data can be patched up afterwards, when the panic is over. INSERTs are generally a little quicker (which adds up over time) because no FKs checks are done. Arbitrary subsets of data can be pulled from the DB as desired without having to ensure all supporting data is included. And so on. This can be absolutely fine if the people involved know the system, take careful notes, have good reconciliation routines, understand the consequences and have the time to tidy up after themselves. The cost is that something, somewhere is missed one time and the database deteriorates into barely creditable junk.
Some teams put the checks in the application rather than in the database. Again, this can work. It seems to me, however, that the total server load will be much the same (or slightly higher) with this approach, but the risk of forgetting some check in a bit of code somewhere is much higher. With DRI the rule's communicated to the computer once and is enforced forever. In application code it has to be re-written with each new program.
For my two cents' worth, I'm all for foreign keys. Let the computers do what they're good at doing, like routine repetitive checking that column values match. We humans can concentrate on dreaming up new and interesting stuff. Having taken responsibility for a system a few months back I'm adding FKs to most tables. There are a few, however, where I will not be adding them. These are where we collect data from external sources. The cost of rejecting these rows is greater than the cost of accepting bad data and fixing it up later. So, for these few tables, there will be no foreign keys. I do this open-eyed, knowing we have monitoring and corrective procedures in place.
1I acknowledge the existence of multi-column foreign key constraints.