perf(forge): backtrace lines at -vvv by recording only terminal opcodes

Open
#17,954 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
12/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
rust, solidity

Research direction

Start at crates/evm/traces/src/backtrace/mod.rs, where each frame reads trace.steps.last(); steps exist only because with_state_changes sets StepRecording::All. The proposal adds a StepRecording::Terminal variant built on the existing record_opcodes_filter (as Jumps does), gated by a new TraceRequirements::with_backtrace_steps that forge test enables at -vvv/-vvvv but not with --json. Done means -vvv backtraces match -vvvvv; tests are backtrace.rs plus two in test_cmd. The author already has a patch ready, so confirm with maintainers first.

Written by the indexing model from the issue text.

Description

A-tracing C-forge T-perf
Component

Forge

Describe the feature you would like

Since #12211 (fix for #12188), backtraces show source lines only at -vvvvv. At -vvv and -vvvv they show contract and function names only:

Backtrace:
  at ProbeVault.withdraw
  at ProbeRouter.route
  at BacktraceProbeTest.test_probe_customErrorNested

The backtrace reads one step per frame: trace.steps.last() of each call on the failing path (crates/evm/traces/src/backtrace/mod.rs). But steps only exist at -vvvvv, where with_state_changes turns on StepRecording::All for every test. That is the cost #12188 hit.

record_opcodes_filter can record just the opcodes a frame can end on, or call out from: STOP, RETURN, REVERT, INVALID, SELFDESTRUCT, CALL, CALLCODE, DELEGATECALL, STATICCALL, CREATE, CREATE2. Foundry already uses the filter for StepRecording::Jumps. I tried a StepRecording::Terminal mode enabled at verbosity 3-4:

  • Same output. On five planted failures (custom error two calls deep, require one call deep, library panic, assertEq, out-of-gas), -vvv printed exactly the backtrace master prints at -vvvvv:

    Backtrace:
      at ProbeVault.withdraw (test/zz/BacktraceProbe.t.sol:18:50)
      at ProbeRouter.route (test/zz/BacktraceProbe.t.sol:42:9)
      at BacktraceProbeTest.test_probe_customErrorNested (test/zz/BacktraceProbe.t.sol:61:9)
    
  • Cost. Measured on a 463-test project suite, 3 runs each, same commit, debug builds:

    Run Wall CPU Max RSS
    -vvv, master 38.0-39.9 s 195-208 s 362-368 MB
    -vvv, with terminal steps 39.5-43.2 s 207-210 s 379-385 MB
    -vvvvv, master 183-195 s 589 s 5.7-6.1 GB

    The -vvvvv row also prints every trace, so it overstates the recording cost alone.

Would you take a PR that records terminal steps in forge test at -vvv/-vvvv, so backtraces get their lines back there? I have it ready: a StepRecording::Terminal variant (merged with Jumps for --decode-internal) and TraceRequirements::with_backtrace_steps, enabled by forge test only, and not with --json or --json-file, because recorded steps are serialized with the traces. So JSON output, forge script, cast run and coverage are unchanged. Existing tests only change where backtraces gain lines (backtrace.rs, two in test_cmd). The library backtrace snapshot at -vvv is identical to a -vvvvv run.

A follow-up, if you want it: the plan in #12188 was to rerun only failing tests to collect backtraces. With terminal steps, that rerun is cheap enough to give one file:line for a failing test at default verbosity, while passing runs pay nothing.

Additional context

Same limit as today: a frame that halts (out of gas) has no step at the halt, so it gets no line; master at -vvvvv gives none either.

Seen on master at -vvvvv, unrelated to this change:

  • With the optimizer on, one test's call line was reported as an identical call in another test, probably shared code.
  • assertEq frames resolve into forge-std rather than the test.
Dominant language
Rust
Stars
10.8k
Forks
2.7k
Avg merge
1d 12m
Merged PRs (30d)
591

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from foundry-rs/foundry

All issues in foundry-rs/foundry

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.