With an average of 5 relationship rows per task per update if you have 1000 tasks created or updated each hour, you will have enough space in an unsigned integer for 49 years.
You could use a BITINT instead (a 64-bit numeric rather than a 32-bit one), though if you are passing the value into any other code that treats it as a numeric type rather than a string or similar you need to ensure that your target language(s) can handle larger numbers natively.
Or as Melvin suggests, use a UUID type (128 bit keyspace, and generally (if you always generate them properly) portable between systems without keyspace conflict).
As a further option: do you actually need a surrogate key on that table? You could not use task+member as a composite key instead if a person will not be linked to the same task twice?
Edit: As for the delete-before-insert (instead of check-and-insert-or-delete-as-needed): there is no significant problem with this if it makes your code simpler and therefore easier to maintain.
It is less efficient as you are removing some rows that will then immediately get put back as they were, so there will be more I/O activity generally and more actions against rows being logged (relevant if you are using differential backups, log shipping, or so forth) and potentially greater fragmentation within the data files (though this latter effect should be small enough to be safely ignored unless there are many updates per second), and you are using more of your keyspace in that INT PK, but often having simpler code is valuable enough to more than be worth these side effects.
Do make sure that your actions are wrapped in a transaction so that if something goes wrong any changes that were made before the problem are all rolled out together instead of leaving your data in an inconsistent state - though you should do this for the other methods of updating multiple rows in different tables anyway.