rust-lang/rust-analyzer

Optimize MBE (macro by example) performance

Open

#7,857 opened on Mar 3, 2021

View on GitHub
 (10 comments) (4 reactions) (0 assignees)Rust (14,924 stars) (1,726 forks)batch import
A-macroA-perfE-has-instructionsS-actionablefungood first issue

Description

Recently (#7513) we implemented a new NFA based expander for MBE which is quite slow compared to the old one (based on recursive descent). It would be nice if someone could optimize it to run as fast as possible.

Benchmark

We already have a benchmark for it, to run the benchmark :

$ RUN_SLOW_TESTS=1 cargo test --release --package mbe -- benchmark::benchmark_expand_macro_rules --nocapture

Output should be something like:

running 1 test
mbe expand macro rules: 2.21s, 8464minstr
test benchmark::benchmark_expand_macro_rules ... ok

Source

The corresponding source code of mbe expander is at :

https://github.com/rust-analyzer/rust-analyzer/blob/3b507aa90fca9618ddbe0667e245ef4766aa96b5/crates/mbe/src/expander/matcher.rs#L148-L151

And we have a bunch of compliance tests related to mbe, to run these test :

$  cargo test --package mbe -- --nocapture

Tips

By default, RA turn off the debug information in Cargo.toml :

https://github.com/rust-analyzer/rust-analyzer/blob/3b507aa90fca9618ddbe0667e245ef4766aa96b5/Cargo.toml#L18-L20

You may need to set it to debug = 2 And here is how I run perf in linux:

$ RUN_SLOW_TESTS=1 perf record --call-graph dwarf  cargo test --release --package mbe -- benchmark::benchmark_expand_macro_rules --nocapture
$ perf report --call-graph

[EDIT: added --release flag]

Contributor guide

Tech stack
rust
Domain
performance
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Rust programmingMacro expansion conceptsPerformance profiling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
15
Research direction
The issue asks to optimize the MBE expander. Start by running the benchmark with 'RUN SLOW TESTS=1 cargo test release package mbe benchmark::benchmark expand macro rules nocapture'. The source code is at crates/mbe/src/expander/matcher.rs lines 148-151. Also run compliance tests with 'cargo test package mbe nocapture'. Use perf for profiling. The debug info may need to be enabled in Cargo.toml.
Optimize MBE (macro by example) performance · rust-lang/rust-analyzer#7857 | Good First Issue