Make LIFO slot stealing opt-in by Darksonn · Pull Request #8092 · tokio-rs/tokio

Darksonn · 2026-04-28T08:16:33Z

We made it possible to steal the LIFO slot in #7431, but this causes a performance regression #8065. Thus, make it opt-in.

The feature is split into two options:

Can stealing happen at all?
Are workers woken up eagerly to check whether stealing should be carried out?

Both options are needed to make the runtime fully tolerate slow tasks, but I split them up as the first option can probably be enabled by default without adverse performance impact. But for now, both are off by default.

Fixes: #8065

Darksonn · 2026-04-28T08:20:42Z

There aren't any changes on master that can't be part of a 1.52.2 release, so merging into master is fine.

martin-g · 2026-04-29T11:18:39Z

+        /// [`enable_eager_lifo_handoff`] option is also enabled.
+        ///
+        /// See the docs for [`disable_lifo_slot`] for more details on what the
+        /// LIFO slot is.


Add some info how the new option(s) depend on the value of disable_lifo_slot ?!
Currently it is possible to do: builder.disable_lifo_slot().enable_lifo_slot_stealing().

Or even add debug_assert!(!self.disable_lifo_slot, "Enabling LIFO slot stealing does not make sense when LIFO slot is disabled");

Updating docs and making these panic.

martin-g · 2026-04-29T11:24:17Z

+            } else {
+                // We pushed to the LIFO slot, and it was empty, so only notify if eager lifo
+                // handoff is enabled.
+                self.shared.config.enable_eager_lifo_handoff


Suggested change

self.shared.config.enable_eager_lifo_handoff

self.shared.config.enable_lifo_stealing && self.shared.config.enable_eager_lifo_handoff

no need to notify if stealing is not enabled, no ?

I'm changing builder to panic in this scenario, so it's not necessary to check both here.

mattiapitossi

Looks good to me

carllerche

I would recommend against specific options for stealing from LIFO etc.. and have a general "wake up more aggressively" option that is used as a basic toggle for the runtime to decide if it should bias for waking up or not.

Darksonn · 2026-05-01T16:20:35Z

I agree but I don't want to do that in a patch release.

carllerche · 2026-05-01T16:42:11Z

Feel free to dismiss my review if it doesn't apply here

Noah-Kennedy · 2026-05-03T20:14:56Z

The implementation looks sound, have approved.

I do somewhat question adding options in a patch but I will not block on that.

Darksonn · 2026-05-04T07:44:43Z

I don't feel that strongly about this, so I'm not going to dismiss your review. I filed #8100 to revert instead.

Darksonn · 2026-05-05T13:46:56Z

I assume Eliza will pick this up when re-enabling this, so closing.

@carllerche

In order to re-enable stealing tasks from the LIFO slot, we would like to allow the user to configure whether or not pushing tasks to the LIFO slot unparks another worker thread. This is in order to avoid performance regressions due to the overhead of additional cross-thread wakeups (i.e. #8065). In [this comment][1] on #8092, @carllerche suggested that we do this by having a single option controlling whether we wake parked workers more or less aggressively which would also control the eager I/O and time driver handoff behavior from #8010, which also causes the runtime to wake parked workers more aggressively in order to prevent potential deadlocks and task starvation. This PR adds such a knob to the `runtime::Builder` type, which requires `tokio_unstable`. This is represented as an `UnparkingMode` enum. We use an enum rather than a `bool` primarily because I would like to add a third unparking mode in the future. This mode would wake parked workers much less frequently, but would instead track whether there are any stealable tasks enqueued, and wake a stealer only when _transitioning_ from 0-1 stealable tasks, and have parking workers park with a timeout while we are in the "stealable tasks" state. The idea is that we would prevent deadlocks while avoiding spurious IPIs in this case. But, I haven't actually implemented this behavior yet, so presently, the enum has two variants: `Traditional`, which is the current runtime behavior, and `Cautious`, which wakes more aggressively to try to prevent deadlocks (and enables waking on LIFO pushes and on I/O driver handoff). The existing `enable_eager_driver_handoff` is deprecated in favor of the new API. I also felt that the enum made it a bit easier to document the different behaviors. If anyone has input on how we can better explain the two unparking modes to users, I'd welcome suggestions! Closes #8118 [1]: #8092 (review)

@carllerche

In order to re-enable stealing tasks from the LIFO slot, we would like to allow the user to configure whether or not pushing tasks to the LIFO slot unparks another worker thread. This is in order to avoid performance regressions due to the overhead of additional cross-thread wakeups (i.e. #8065). In [this comment][1] on #8092, @carllerche suggested that we do this by having a single option controlling whether we wake parked workers more or less aggressively which would also control the eager I/O and time driver handoff behavior from #8010, which also causes the runtime to wake parked workers more aggressively in order to prevent potential deadlocks and task starvation. This PR adds such a knob to the `runtime::Builder` type, which requires `tokio_unstable`. This is represented as an `UnparkingMode` enum. We use an enum rather than a `bool` primarily because I would like to add a third unparking mode in the future. This mode would wake parked workers much less frequently, but would instead track whether there are any stealable tasks enqueued, and wake a stealer only when _transitioning_ from 0-1 stealable tasks, and have parking workers park with a timeout while we are in the "stealable tasks" state. The idea is that we would prevent deadlocks while avoiding spurious IPIs in this case. But, I haven't actually implemented this behavior yet, so presently, the enum has two variants: `Traditional`, which is the current runtime behavior, and `Cautious`, which wakes more aggressively to try to prevent deadlocks (and enables waking on LIFO pushes and on I/O driver handoff). The existing `enable_eager_driver_handoff` is deprecated in favor of the new API. I also felt that the enum made it a bit easier to document the different behaviors. If anyone has input on how we can better explain the two unparking modes to users, I'd welcome suggestions! Closes #8118 [1]: #8092 (review)

Darksonn requested a review from hawkw April 28, 2026 08:16

Darksonn added A-tokio Area: The main tokio crate M-runtime Module: tokio/runtime labels Apr 28, 2026

github-actions Bot added the R-loom-multi-thread Run loom multi-thread tests on this PR label Apr 28, 2026

Darksonn mentioned this pull request Apr 28, 2026

Performance regression on high-QPS short-handler workloads since 1.51.0 #8065

Closed

Darksonn force-pushed the lifo-stealing-optional branch from 4ba387c to bafc878 Compare April 28, 2026 08:19

github-actions Bot added the R-loom-time-driver Run loom time driver tests on this PR label Apr 28, 2026

Darksonn commented Apr 28, 2026

View reviewed changes

Comment thread tokio/src/runtime/time/wheel/mod.rs

runtime: make LIFO slot stealing opt-in

6add0b7

Darksonn force-pushed the lifo-stealing-optional branch from b874b10 to 6add0b7 Compare April 29, 2026 06:01

github-actions Bot removed the R-loom-time-driver Run loom time driver tests on this PR label Apr 29, 2026

mattiapitossi reviewed Apr 29, 2026

View reviewed changes

Comment thread tokio/src/runtime/builder.rs

martin-g reviewed Apr 29, 2026

View reviewed changes

thieman mentioned this pull request Apr 29, 2026

chore(agent-data-plane): update to tokio 1.52.1 and re-enable eager driver handoff DataDog/saluki#1411

Closed

2 tasks

Darksonn changed the base branch from master to tokio-1.52.x April 30, 2026 08:51

Darksonn added 3 commits April 30, 2026 10:59

Address review

67c8919

Add loom test with lifo stealing

ec8540f

Update can_steal docs

d94c49d

mattiapitossi reviewed Apr 30, 2026

View reviewed changes

Comment thread tokio/src/runtime/config.rs Outdated

Adjust wording in config.rs

0a055a3

mattiapitossi approved these changes May 1, 2026

View reviewed changes

carllerche requested changes May 1, 2026

View reviewed changes

Noah-Kennedy approved these changes May 3, 2026

View reviewed changes

Darksonn closed this May 5, 2026

hawkw mentioned this pull request May 5, 2026

re-enable LIFO slot stealing #8120

Open

	self.shared.config.enable_eager_lifo_handoff
	self.shared.config.enable_lifo_stealing && self.shared.config.enable_eager_lifo_handoff

Uh oh!

Conversation

Darksonn commented Apr 28, 2026

Uh oh!

Darksonn commented Apr 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

martin-g Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Darksonn Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

martin-g Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Darksonn Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mattiapitossi left a comment

Choose a reason for hiding this comment

Uh oh!

carllerche left a comment

Choose a reason for hiding this comment

Uh oh!

Darksonn commented May 1, 2026

Uh oh!

carllerche commented May 1, 2026

Uh oh!

Noah-Kennedy commented May 3, 2026

Uh oh!

Darksonn commented May 4, 2026

Uh oh!

Darksonn commented May 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants