`node-bencode` can produce dictionary entries with duplicate keys.

Open
#146 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript, node.js
Domain
backend

Research direction

Start with lib/encode.js lines 53-55 and the linked uint8-util _node.js implementation; run the provided lone-surrogate proof of concept. Trace how the JavaScript keys become UTF-8 bytes and add a regression check for the reported object. Done means encoding it no longer emits dictionary entries with duplicate keys.

Written by the indexing model from the issue text.

Description

Bug

node-bencode can produce dictionary entries with duplicate keys.

node-bencode assumes that binary string keys made out of unique Javascript string keys are unique as well, which is false.

https://github.com/webtorrent/node-bencode/blob/ee70f267c8d34b9a94820ca8c42cd67d1274fc89/lib/encode.js#L53-L55
https://github.com/ThaUnknown/uint8-util/blob/149c44c010b3ad17a7904c4266545bbca1fd4403/_node.js#L13

 encode.string = function (buffers, data) { 
   buffers.push(text2arr(text2arr(data).byteLength + ':' + data)) 
 } 

export const text2arr = str => new Uint8Array(Buffer.from(str, 'utf8'))

Proof-of-concept

For example, let node-bencode try encoding {"\uD800": 1, "\uDFFF": 2}. It’ll produce dictionary entries with the duplicate key, "3:\xEF\xBF\xBD".

const lone_surrogates = "\uD800\uDFFF";
// Lone (“unmatched”) UTF-16 surrogates. Invalid in UTF-16.

const a = Buffer.from(lone_surrogates[0], "UTF-8");
const b = Buffer.from(lone_surrogates[1], "UTF-8");
// Decoding the Javascript strings in UTF-16 and encoding them into UTF-8.

console.log(a, a.toString(), b, b.toString());
//  Since those Javascript strings are invalid in UTF-16,
// those lone surrogates are decoded
// into `REPLACEMENT CHARACTER`s (U+FFFD)
// and subsequently encoded into `<Buffer ef bf bd>`.
// Meaning,

console.log(a.equals(b));
// is true, when (lone_surrogates[0] === lone_surrogates[1]) is false.
Dominant language
JavaScript
Stars
173
Forks
38
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from webtorrent/node-bencode

All issues in webtorrent/node-bencode

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.