perf: reduce per-request FFI round-trip cost
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 45/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- cpp
- Domain
- backend-api-design, performance
Research direction
Start by benchmarking the sequential /hello and load cases described in the issue, then read the request path around RawIncomingBatch, Combiner, packedHeaders, FastCalls, and Dart_PostCObject. A contribution is complete when one listed optimization is implemented and the benchmarks show its effect without changing request or response behavior.
Written by the indexing model from the issue text.
Description
Problem
Per-request latency is dominated by the FFI round-trip: the engine posts the request head to Dart via Dart_PostCObject (async, event-loop scheduled), the handler runs, then respond crosses back. This ~async hop is why nitro's sequential /hello (~76 µs) trails dart:io (~67 µs) and why load latency (~977 µs @ 64 conns) sits above dart:io (~816 µs). dart:io avoids the boundary by handling requests in-isolate.
Already done
- Request headers packed into one string, unpacked lazily (
packedHeaders). - Query parsed on first access.
- Per-isolate head batching (
RawIncomingBatch+ flat-combiningCombiner): heads that arrive while the previous post is in flight cross as one message. - Sync handlers answered inline (no extra
Future). - Leaf-call FFI (
FastCalls, ~0.18 µs/call). - Metrics bucket resolved once at dispatch (removed a redundant
_entryForper request on the answer path).
Addressable next steps
- Head as one zero-copy blob; decode method/path/params on access (headers + query already lazy). Expected small.
- Pre-encoded response heads for
constheader maps (skip re-encoding per answer). - Investigate a lower-latency C++→Dart signal than one
Dart_PostCObjectper batch (this is anitrobridge concern; may not be addressable here).
These reduce the Dart-side critical path; the async port hop itself is inherent to the C++-engine + Dart-handler design and is the harder ceiling (see the reactor issue).
- Dominant language
- Dart
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Shreemanarjun/nitro_server
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
Shreemanarjun/nitro_server#1 · 1 comment ·
All issues in Shreemanarjun/nitro_server
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
PhoebeSoftware/xrooster#41 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
VeryGoodOpenSource/very_good_templates#665 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
VeryGoodOpenSource/very_good_cli#1783 ·
Maintainers usually reply within 1 day
-
bug to check
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 76/100
aaronified/fluenough#665 ·
Maintainers usually reply within 1 day