Length-delimited framing actually emitting big-endian length, not little-endian
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 62/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- rust
- Domain
- documentation
Research direction
Search the repository documentation for the length_delimited framing text under sinks, then compare it with the referenced lib/codecs/src/encoding/framing/length_delimited.rs implementation and the tokio-util codec documentation. Done means the documented byte order matches the big-endian framing actually emitted by LengthDelimitedCodec.
Written by the indexing model from the issue text.
Description
A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
The docs for framing in sinks say:
length_delimited
Event data is prefixed with its length in bytes.
The prefix is a 32-bit unsigned integer, little endian.
Where in fact this framing emits Big Endian-encoded integer.
Documentation needs to be fixed.
See here, the tokio_util::codec::LengthDelimitedCodec codec is used:
https://github.com/vectordotdev/vector/blob/7cf2f009dbd9be4177dfbce7950cd82d57f93448/lib/codecs/src/encoding/framing/length_delimited.rs#L25
And this codec emits big-endian: https://docs.rs/tokio-util/latest/tokio_util/codec/length_delimited/#getting-started
The returned transport implements Sink + Stream for BytesMut. It encodes the frame with a big-endian u32 header denoting the frame payload length:
I've confirmed it when I was writing a server to receive Vector events.
Configuration
socket:
type: socket
inputs:
- parse_logs
mode: unix
path: /tmp/foo.socket
encoding:
codec: text
framing:
method: length_delimited
healthcheck:
enabled: false
Version
0.35
Debug Output
No response
Example Data
No response
Additional Context
No response
References
No response
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 137
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from vectordotdev/vector
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
vectordotdev/vector#26389 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vectordotdev/vector#26353 · 1 comment ·
-
source: nats
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
vectordotdev/vector#26345 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vectordotdev/vector#26115 ·
-
domain: buffers
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
vectordotdev/vector#25440 · 1 comment ·
All issues in vectordotdev/vector
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
gitbutlerapp/gitbutler#15998 · 1 comment ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
bitcoindevkit/bdk-ffi#1125 ·