Can someone explain what bloating means in database terms? For example what does it mean to say an index is bloated. I tried searching for it but there is no explanation of what bloating is, only what it causes or what it is caused by.
Due to how PostgreSQL handles transactions and concurrency, MVCC - Multi-Version Concurrency Control, you can get bloat. In PostgreSQL, when you do an
DELETE, the row isn't actually physically deleted. For a
DELETE, it simply marks the row as unavailable for future transactions, and for
UPDATE, under the hood it's a combined
DELETE, where the previous version of the row is marked unavailable.
While the data is marked unavailable, it is still there, and the space cannot be used. To then mark the space as available for use by the database, a vacuum process needs to come along behind the operations, and mark that space available for the database to use. It isn't returned to the operating system, however. That only happens when there are no active rows in an entire page, which can be uncommon in some workloads. This can be a good thing for some workloads, because you can just simply update the space on the individual pages inside of the data files, without needing to add additional data files.
Problems come about with bloat when there are are excessively large numbers of dead tuples versus live tuples. Walking along and checking all of the visibility flags takes time, and having more data files for a relation results in additional unnecessary IO load. Bloat is especially noticeable on indexes, which can also have many dead tuples, sometimes many more than the table. Bloat can slow index lookups and scans, which will show up in slowly increasing query times and changing query plans.
You can restore space by using pg_reorg, pg_repack,
VACUUM FULL. This will go through and reorganize the files, moving tuples and reorganizing to make sure that there are no dead tuples, which will eliminate the bloat.
Bloat can also be efficiently managed by adjusting
VACUUM settings per table, which marks dead tuple space available for reuse by subsequent queries.
You can use queries on the PostgreSQL Wiki related to Show Database Bloat and Index Bloat to determine how much bloat you have, and from there, do a bit of performance analysis to see if you have problems with the amount of bloat you have on your tables.
This probably refers to common issues with indexes where either two many columns are added to the index or there are overlapping indexes on a table. That is, multiple indexes with the same set of columns in them (making one of them unnecessary). I would review all indexes on tables looking for overlapping columns, looking for indexes which are just subsets of other indexes and remove them IF you can determine that they aren't being used.
On top of that, as data in tables are updated, the indexes can become fragmented which causes them to be larger than necessary. I'm not familiar with postgres, but I suspect there are methods to defragment the indexes (by rebuilding the indexes) that will reduce their size on disk.