I agree that the burden of justification should be on the ones requiring access. Typically in environments where I have consulted, I have had access to production systems where it was a small environment and I was the support person. I have had access to backups, etc. where I was support for the support, and indirect access (through a dedicated support developer) to production data.
The big thing is: You need this access when you are on the hook for keeping everything running smoothly and you have to answer the finance guy's question about something not working. You can't always work from even day-old data in that case. On the other hand, the more access the worse it is. Typically as a consultant I tend to avoid getting this sort of access unless it is needed. Since I am working on financial databases, the last thing I want is to be accused of entering my own invoices :-D.
On the other hand, if you don't need access you shouldn't have it. I don't really buy the sensitive data argument since the developer is probably on the hook for making sure this is handled correctly (and it is very hard to verify without looking at what has actually been stored when a bug report comes in). If you can't trust the developer to look at the data the developer's app is storing, you shouldn't hire the developer to write the app. There are too many ways the developer could obfuscate the data and email it away and you can never be sure. MAC controls help here but they are still pretty complex to implement.
The big issue from my side has to do with write access. If a developer has no access then, a fortiori, the developer has no write access. If you want to verify the integrity of the books, you want to keep write access to as few people as you can. Audit trails are far easier to validate if the developers have no access. If the developer has read access, then you always have some question as to whether there has been some privilege escallation attach that can give write access (maybe in-stored-procedure SQL injection?). I have often had full access to client billing info when I have had access to staging environments. If there is a staging environment that works though, I will usually actively ask not to have access to production unless it is necessary.
So this isn't perfect, of course. A developer could still build back-doors into the application that may not be readily detectable, but this approach is a reasonable approach, given the fact that backup data is available from a day prior it seems to me that this is the concern they have.
Hope this helps.
Edit: Just adding that on the larger environments I have worked in, I have had access to full backup data often ranging from a few days old to a few months old for the finance system. This has always been good enough for my work and the only times it has broken down have been when the finance guys needed an ability to test with newer data so they could match against production.