mre/hyperjson

Speed up boolean encoding/decoding

Open

#68 opened on Sep 21, 2019

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (41 forks)github user discovery
enhancementgood first issuehacktoberfesthelp wanted

Repository metrics

Stars
 (511 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

From our benchmarks we can see that we are consistently slower than everyone else when serializing/deserializing boolean values. We should fix that.

orjson is using an unsafe block to create a reference to a boolean: https://github.com/ijl/orjson/blob/03d55e99a953ce93cedc05f03e4b63b0bcbbcc7a/src/decode.rs#L81-L96

This avoids additional allocations. For comparison, this is our code at the moment:

https://github.com/mre/hyperjson/blob/ded13b4100638aa32fe19dc477f5cfe3e704893c/src/lib.rs#L475-L480

I wonder if we could achieve comparable performance without using unsafe. @konstin, any idea? Maybe there was a recent development in pyo3 that we could leverage here?

Contributor guide

Research direction
Investigate benchmark differences between hyperjson and orjson for boolean serialization. Examine orjson's unsafe approach and explore safe alternatives using PyO3 or Rust's standard library to avoid allocations.
Tech stack
pythonrust
Domain
backend
Issue type
Performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonRust
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50