I was able to get this fixed using a bit of a hack. Basically, drop the table from the publication, refresh the publication from the subscriber, add the table to the publication again, then refresh the publication again, but with
drop the table from the publication.
alter publication my-publication drop table only my-table ;
Refresh the subscription.
ALTER SUBSCRIPTION my-subscription REFRESH PUBLICATION;
The log lines seem to stop at this point. Good!
Add the table to the publication.
alter publication my-publication add table only my-table ;
Refresh the subscription with
ALTER SUBSCRIPTION my-subscription REFRESH PUBLICATION with (copy_data = false);
This seems to work, though it seems like it'd be problematic on a table that was changing rapidly.
It'd be nice to put this in a transaction somehow, but the commands are being run alternately on the publisher and the subscriber, so I don't know how that'd work.