ThrottledLifoSem is a semaphore that can wait up to a configurable wakeUpInterval before waking up a sleeping waiter. This gives an opportunity to new waiters to consume the posted values, avoiding the overhead of waking up a thread when the already active threads can consume values fast enough, effectively allowing to batch the work. The semaphore is "throttled" because sleeping waiters can be awoken at most once every wakeUpInterval.
Declared in <folly/synchronization/ThrottledLifoSem.h>
class ThrottledLifoSem;The motivating example is the task queue of a thread pool, where if a burst of very small tasks (order of hundreds of nanoseconds each) is enqueued, it may be beneficial to have a single thread process all the tasks sequentially, rather than pay for the cost of waking up all the idle threads. However, if nothing consumes the posted value, more waiters are awoken, each after a wakeUpInterval delay. Sleeping waiters are awoken in LIFO order, consistently with LifoSem.
This is realized by having at most one sleeping waiter being in a "waking" state: when such waiter is awoken, it immediately goes to sleep until last wakeup time + wakeUpInterval to allow more value to accumulate and other threads to consume it. If at the end of the sleep no value is left, as it was consumed by other thread, the waking waiter can go back to sleep. Otherwise, a new waiter becomes waking (if necessary) and control is returned to the caller.
Note that since wakeUpInterval is relative to the last awake time, in the regime where post()s are spaced at least wakeUpInterval apart the waiters are always awoken immediately, so there is no added latency in this case. Also, when there is more outstanding value than waiters (for example, the thread pool is saturated), there is no added latency compared to LifoSem.
The interface is a subset of LifoSem, with semantics compatible with it. Only the minimal subset needed to support task queues is currently implemented, but the interface can be extended as needed.
| Name | Description |
|---|---|
Options | Configuration options for the semaphore. |
| Name | Description |
|---|---|
ThrottledLifoSem [constructor] | Constructors |
~ThrottledLifoSem [destructor] | Destroys the semaphore, asserting that no waiters remain. |
excessValueGuess | Returns a best-effort estimate of the value not yet claimed by waiters. |
post | Posts the given value to the semaphore. |
try_post | Conditionally posts the given value if enough waiters are present. |
try_wait | Attempts to decrement the semaphore without blocking. |
try_wait_for | Waits until the semaphore can be decremented or the timeout elapses. |
try_wait_until | Waits until the semaphore can be decremented or the deadline expires. |
valueGuess | Returns a best-effort estimate of the current value. |
wait | Blocks until the semaphore can be decremented. |