There are multiple problems.
Auto_inc of 922M is half way to the 2 billion limit on INT SIGNED. Suggest you change to INT UNSIGNED (4 billion limit) during the next ALTER.
MyISAM saves disk space, but is otherwise 'worse' than InnoDB. Note: changing to InnoDB will require changing several settings.
FOREIGN KEYS are ignored by MyISAM.
If sha is a SHA-1 hash, it is terrible for indexing.
If sha is a SHA-1 hash, it could be compressed to BINARY(20) via UNHEX(). This would shrink the table by over 20GB, 30% of the current size!
If sha is a SHA-1 hash, don't use utf8; use ascii or latin1.
If sha is a SHA-1 hash, and that is the column you are creating the UNIQUE index on, check SHOW PROCESSLIST. If it says "Repairing by key_buffer", then you should kill it; it will take months to finish. If it says "Repairing by sort", then there is hope that it will finish.
Consider using smaller ids than the 4-byte INT for author, committer, and project -- unless you really have many billion distinct values. Wait! What? Each of those is going to be UNIQUE? I doubt it.
What SELECTs do you have? Might some of them need 'composite' indexes?
Put multiple ALTERs (including creating indexes) into a single statement. Each ALTER is a complete copy of the table (in MyISAM).
myisam_sort_buffer_size = 8G is half of RAM. This is bad. Suggest 3G.
Polynomial
The trend looks polynomial.
I hope you are not adding a UNIQUE INDEX on a column on a table that already has the same column as PRIMARY KEY. That would be totally redundant. Please provide SHOW CREATE TABLE for one of them.
Why "polynomial"? If it is (and I question such), then it would be because of the nature of "Repair by key_buffer":
For each row, look up the column in all unique (including primary) indexes to make sure it is not a "duplicate". This lookup requires fetching the block of the index's BTree and putting it into the "key_buffer" for testing. Depending on the 'order' of the key versus the 'order' of the data:
- For auto_increment or timestamp, where the row were inserted chronological (and the table had not become fragmented), the next block needed is very likely to be the one that was just touched. This is the most efficient because it needs the least I/O.
- For SHA1/MD5/UUID/etc, this lookup will be jumping all around the index. So the next block needed is less and less likely to be in the in-RAM key_buffer. At the extreme, this leads to almost 1 disk read per lookup!
- For other indexes, the timing is somewhere in between.