"there is no reactor running" panic when system is quickly dropped · actix/actix#531

(5 comments) (4 reactions) (0 assignees)Rust (611 forks)batch import

A-actixC-bughelp wanted

Repository metrics

Stars: (7,757 stars)
PR merge metrics: (Avg merge 12d 13h) (5 merged PRs in 30d)

Description

Here is code that reproduces the issue: main.rs

struct A {}

impl actix::Actor for A {
    type Context = actix::Context<Self>;

    fn started(&mut self, ctx: &mut Self::Context) {
        use actix::AsyncContext;
        ctx.run_interval(std::time::Duration::from_millis(100), |_a, _c| {
            println!("Hi!");
        });
    }
}

fn main() {
    actix_rt::System::new().block_on(async {
        actix::Actor::start(A {});
        // tokio::time::sleep(std::time::Duration::from_millis(1)).await;
    });
}

Cargo.toml

[dependencies]
actix = "0.13.0"
actix-rt = "2.7.0"
tokio = { version = "1.17.0", features = ["time"]}

Expected Behavior

The program does not panic. Nothing should be sent to stdout, as system is dropped before 100 milliseconds pass (interval).

Current Behavior

The program panics and produces the following backtrace on crash:

thread 'main' panicked at 'there is no reactor running, must be called from the context of a Tokio 1.x runtime', /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/context.rs:54:26
stack backtrace:
   0: rust_begin_unwind
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/panicking.rs:101:14
   2: core::option::expect_failed
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/option.rs:1615:5
   3: core::option::Option<T>::expect
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/option.rs:698:21
   4: tokio::runtime::context::time_handle::{{closure}}
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/context.rs:54:13
   5: std::thread::local::LocalKey<T>::try_with
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/thread/local.rs:399:16
   6: tokio::runtime::context::time_handle
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/context.rs:52:15
   7: tokio::time::driver::handle::Handle::current
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/time/driver/handle.rs:57:13
   8: tokio::time::driver::sleep::Sleep::new_timeout
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/time/driver/sleep.rs:257:22
   9: tokio::time::driver::sleep::sleep
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/time/driver/sleep.rs:129:27
  10: actix::utils::IntervalFunc<A>::new
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.13.0/src/utils.rs:186:20
  11: actix::actor::AsyncContext::run_interval
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.13.0/src/actor.rs:459:20
  12: <playround::A as actix::actor::Actor>::started
             at ./src/main.rs:8:9
  13: <actix::contextimpl::ContextFut<A,C> as core::future::future::Future>::poll
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.13.0/src/contextimpl.rs:360:13
  14: <actix::contextimpl::ContextFut<A,C> as core::ops::drop::Drop>::drop
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.13.0/src/contextimpl.rs:226:21
  15: core::ptr::drop_in_place<actix::contextimpl::ContextFut<playround::A,actix::context::Context<playround::A>>>
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ptr/mod.rs:188:1
  16: core::ptr::drop_in_place<tokio::runtime::task::core::Stage<actix::contextimpl::ContextFut<playround::A,actix::context::Context<playround::A>>>>
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ptr/mod.rs:188:1
  17: tokio::runtime::task::core::CoreStage<T>::set_stage::{{closure}}
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:214:35
  18: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/loom/std/unsafe_cell.rs:14:9
  19: tokio::runtime::task::core::CoreStage<T>::set_stage
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:214:9
  20: tokio::runtime::task::core::CoreStage<T>::drop_future_or_output
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:180:13
  21: tokio::runtime::task::harness::cancel_task::{{closure}}
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:438:9
  22: core::ops::function::FnOnce::call_once
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ops/function.rs:227:5
  23: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/panic/unwind_safe.rs:271:9
  24: std::panicking::try::do_call
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/panicking.rs:403:40
  25: __rust_try
  26: std::panicking::try
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/panicking.rs:367:19
  27: std::panic::catch_unwind
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/panic.rs:129:14
  28: tokio::runtime::task::harness::cancel_task
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:437:15
  29: tokio::runtime::task::harness::Harness<T,S>::shutdown
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:147:9
  30: tokio::runtime::task::raw::shutdown
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:164:5
  31: tokio::runtime::task::raw::RawTask::shutdown
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:109:18
  32: tokio::runtime::task::Task<S>::shutdown
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/mod.rs:338:9
  33: tokio::runtime::task::list::LocalOwnedTasks<S>::close_and_shutdown_all
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/list.rs:225:13
  34: <tokio::task::local::LocalSet as core::ops::drop::Drop>::drop::{{closure}}
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:606:13
  35: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/macros/scoped_tls.rs:61:9
  36: tokio::task::local::LocalSet::with
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:561:9
  37: <tokio::task::local::LocalSet as core::ops::drop::Drop>::drop
             at /home/prk3/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:603:9
  38: core::ptr::drop_in_place<tokio::task::local::LocalSet>
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ptr/mod.rs:188:1
  39: core::ptr::drop_in_place<actix_rt::runtime::Runtime>
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ptr/mod.rs:188:1
  40: core::ptr::drop_in_place<actix_rt::system::SystemRunner>
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ptr/mod.rs:188:1
  41: playround::main
             at ./src/main.rs:18:7
  42: core::ops::function::FnOnce::call_once
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ops/function.rs:227:5

Possible Solution

I suspect run_interval creates some task that isn't dropped in time or it always assumes tokio runtime exists. I noticed that a short sleep just after staring the actor fixes the issue (commented in the example). Explicit ctx.cancel_future in stopping or stopped method does not fix the problem.

Steps to Reproduce (for bugs)

Call run_interval in actor's started method.
Start actor in actix-rt system.
Quickly drop the system.

Context

We came across this error twice. Once when running a test function with should_panic attribute. It looks like the expected panic triggers drops and the drop of the system causes this panic. Another case was starting and dropping a system in a loop in a fuzz test.

Your Environment

Linux work 5.16.13-200.fc35.x86_64 #1 SMP PREEMPT Tue Mar 8 22:50:58 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Rust Version (I.e, output of rustc -V): rustc 1.58.1 (db9d1b20b 2022-01-20)
Actix Version: 0.13.0, 0.12.0

Contributor guide

Research direction: Trace the drop order of the actor context and ensure that the tokio time handle is not used after the runtime is dropped. Check if run interval schedules a timer that requires the runtime to be alive during drop.
Tech stack: rust
Domain: backend
Issue type: Bug
Difficulty: 2
Estimated time: 1-3 hours
Activity status: Active
Clarity: Clear
Prerequisites: Rust
Newbie friendliness: 70