The official mongodb documentation states to use three config servers. They say its for redudancy, because 1 server would be a point of failure.... but why not 2? in a production sharded cluster, if I have 1 shard, I shouldn't need more than 2, right? Is one config server only meant to handle 1 shard? so, if the minimum is 2 shards, then at least I should have 3?
Config servers are a special case, you can use one (for testing), or three and that's it. They are not a replica set, at least not yet, that might change in the future, see SERVER-1448.
Go and (re)read the first two paragraphs here - note the piece about them not being a replica set is explicitly mentioned. I will pick out some other relevant points:
Config servers use a two-phase commit to ensure immediate consistency and reliability
to ensure redundancy and safety in production, you should always use three
The first part gives you a hint about why 3 are needed, the second part is telling, particularly in terms of the safety aspect (redundancy is obvious - you have 3 copies of the data).
With how the 2 phase commit is implemented, all three config servers must receive (and acknowledge) a change for it to be considered successful. That's the safety piece - all three must be in a quorum and have the same data (there are regular checks), and that is why it is three config servers required exactly (for now). When you run with just one for testing, this can't happen, and your cluster meta data is essentially at risk, hence using just one is for testing purposes only.
The document does not say why we need 3 config servers. But it is recommended to use odd number of mongods in each shard, maybe for quick elect? I think it is the same reason why we need odd number of config servers. I am not sure.
Each config server can handle lots of shards. config server does not consume too much resource, such as disk. Because config server store config info, meta data, not the user data. So you can use one config server in your deployment for lots of shards. But in production, for redundancy, it is suggested to use 3 config servers. each config server has nearly the same data copy.
Why it always 1 or more than 2? Obvious answer is split brain problem. Any instance or programs that falls under replication would face this issue. Read more this