O(1) cancelation for Semaphore and MiniSemaphore

[stasimus]

Fixes #4434. Cancelation no longer filters the waiter queue (O(n) under a CAS loop, O(n²) on mass cancelation); a canceled waiter races its gate and is lazily swept by releases. Trade-offs:

1. Canceled waiters linger in the queue until a release reaches them.
2. count may transiently include them.
3. Each such waiter costs one extra release pass.

[djspiewak]

I quite like this! Thank you for taking this on.

Intuitively, cancelation should always be pretty drastically less frequent than releasing, but I think it is the case that, when it happens, it is frequently a large scale event, particularly with MiniSemaphore. Thus, amortizing the cancelation cleanup cost into the release makes sense. This also has the neat side benefit in the parTraverseN case of making cancelation free, since we're throwing the whole semaphore away in a moment.

What this doesn't fix is the false starting that occurs in mass cancelation cases in parTraverseN, so while I'm certain that this drastically improves the linked issue, it doesn't close it.

Do you mind if we land this in series/3.6.x? You'll need to rebase and reparent this PR.

[reardonj]

Do you mind if we land this in series/3.6.x? You'll need to rebase and reparent this PR.

@djspiewak , 3.7.0 is latest. Is there actually going to be another 3.6.x release?

[djspiewak]

@reardonj Just one more. Mostly I'm trying to get parTraverseN in. We can target this at 3.7 instead if it's easier (that way we don't have to backport the unit tests).