Foundry panics if block is generated faster than 1 block/sec

core/src/consensus/tendermint/worker.rs から始め、特に 1567 行目の broadcast_proposal_block と 1320 行目の on_timeout を確認してください。1 秒あたり 1 ブロックを超える速度でブロック生成を再現し、proposal のタイムスタンプ処理を追跡してください。完了条件は、高速なブロック生成で「Proposal vote is generated before」panic に到達しなくなり、生成されたブロックが有効なままであることです。

Background:

• Foundry does not allow blocks that have the same timestamp.
• Foundry allows blocks from the future, up to 5 seconds. For instance, if the current timestamp is A, a block with the timestamp A + 7 will be rejected.
• When creating a block, the timestamp of the block will be min(parent block's timestamp + 1, current timestamp).

What happened:
If a node creates blocks faster than 1 block/sec, it will eventually generate blocks with future timestamps. We are assuming that if the foundry itself creates a block, it is always valid. Since a block that has a future timestamp violates the assumption, foundry stops at a strange point.

Here is the panic log:

 
 ====================
 
 stack backtrace:
    0:     0x55e341b60163 - backtrace::backtrace::libunwind::trace::h78e882580cf0470d
                         at /home/juhyung/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.7/src/backtrace/libunwind.rs:53
                          - backtrace::backtrace::trace::he884cb33efaf1cc8
                         at /home/juhyung/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.7/src/backtrace/mod.rs:42
    1:     0x55e341b51643 - backtrace::capture::Backtrace::new_unresolved::h2195345b6bfc30d0
                         at /home/juhyung/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.7/src/capture.rs:88
    2:     0x55e341b5159d - backtrace::capture::Backtrace::new::h364652a813c7f694
                         at /home/juhyung/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.7/src/capture.rs:63
    3:     0x55e3400dc2ed - panic_hook::panic_message::h50fdd7579f1d10ec
                         at util/panic_hook/src/lib.rs:75
    4:     0x55e3400dbb7a - panic_hook::panic_hook::hbdf778282053c9fe
                         at util/panic_hook/src/lib.rs:44
    5:     0x55e3400de8a7 - core::ops::function::Fn::call::h74b5112062c5f02a
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libcore/ops/function.rs:69
    6:     0x55e341eb782f - std::panicking::rust_panic_with_hook::hbe174577402a475d
                         at src/libstd/panicking.rs:468
    7:     0x55e341eb72fd - std::panicking::continue_panic_fmt::h4d855dad868accf3
                         at src/libstd/panicking.rs:373
    8:     0x55e341eb71e5 - rust_begin_unwind
                         at src/libstd/panicking.rs:302
    9:     0x55e341ed89ad - core::panicking::panic_fmt::hdeb7979ab6591473
                         at src/libcore/panicking.rs:139
   10:     0x55e341ed8a16 - core::option::expect_failed::h6c3ec3658ee2670d
                         at src/libcore/option.rs:1185
   11:     0x55e3405da3c1 - core::option::Option<T>::expect::h197b26790b12cd05
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libcore/option.rs:345
   12:     0x55e34094c9cf - codechain_core::consensus::tendermint::worker::Worker::broadcast_proposal_block::he7fd2a2fc5a7e5d4
                         at core/src/consensus/tendermint/worker.rs:1567
   13:     0x55e3409491b3 - codechain_core::consensus::tendermint::worker::Worker::on_timeout::h00be05dcbfa25fba
                         at core/src/consensus/tendermint/worker.rs:1320
   14:     0x55e340803558 - codechain_core::consensus::tendermint::worker::Worker::spawn::{{closure}}::h0daa882c0626e5cd
                         at core/src/consensus/tendermint/worker.rs:320
   15:     0x55e340935b81 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9b01b58ab4cc8ed9
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/sys_common/backtrace.rs:129
   16:     0x55e3408f02a0 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hab353df2b7b7e832
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/thread/mod.rs:469
   17:     0x55e34091fbb0 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h5452be344df113f7
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/panic.rs:317
   18:     0x55e34096b0cd - std::panicking::try::do_call::h92eebfceeca86128
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/panicking.rs:287
   19:     0x55e341ec0579 - __rust_maybe_catch_panic
                         at src/libpanic_unwind/lib.rs:78
   20:     0x55e34096a557 - std::panicking::try::h632d8b03394d7d4e
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/panicking.rs:265
   21:     0x55e34091fdf2 - std::panic::catch_unwind::had57540be874eeec
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/panic.rs:396
   22:     0x55e3408ef955 - std::thread::Builder::spawn_unchecked::{{closure}}::ha85ccfc2020068e0
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/thread/mod.rs:468
   23:     0x55e3404103e3 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hc4b4c1b1a2fc8a6c
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libcore/ops/function.rs:227
   24:     0x55e341ea56de - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h3534c64212330b0c
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
   25:     0x55e341ebf63f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h338c10574a337ece
                         at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
                          - std::sys_common::thread::start_thread::h761ac6d57710d65d
                         at src/libstd/sys_common/thread.rs:13
                          - std::sys::unix::thread::Thread::new::thread_start::h61c012ef60f933c0
                         at src/libstd/sys/unix/thread.rs:79
   26:     0x7f63f973b668 - start_thread
   27:     0x7f63f9647322 - clone
   28:                0x0 - <unknown>
 
 Thread 'tendermint' panicked at 'Proposal vote is generated before', src/libcore/option.rs:1185
 
 This is a bug. Please report it at:
 
     https://github.com/CodeChain-io/codechain/issues/new