Arbitrary Strings Not Properly Encodable.

Open
#94 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Domain
testing

Research direction

Start by locating the Arbitrary instances for strings and characters in the repository, then inspect how generated UTF-16 code units are used by the existing tests. The issue proposes generating strings from code points or restricting the character range to U+0000–U+D7FF; determine which approach fits the library and verify the encoding behavior with the reported text encoding work.

Written by the indexing model from the issue text.

Description

type: enhancement

Hi Folks, I have ongoing work to deal with string encoding for the purpose of supporting cryptography in purescript, and I am attempting to isolate the cause of the problem I encounter.

I suspect currently that the cause of the problem is actually in the way that quickcheck generates strings via Arbitrary. That is to say, that generating randomly ordered CodeUnits (purescript Char), is not truly compliant with the way UTF-16 strings should be encoded. If we wanted to join randomly generated strings, these should be the (potenitially) multi code-unit strings, aka CodePoints.

I think the best solution would be to redefine arbitrary string such that it is generated from Code Points, not Code Units. Failing this however, a "quick and dirty" solution would be to redefine the char range used to generate strings to always fall within the U+0000 to U+D7FF range. (Represented identically between their code point and code unit).

I will be happy over the weekend to write the PR should I find time and there are no objections raised here. It will also be used to verify my theory by using it with existing text encoding work I've done.

Dominant language
PureScript
Stars
108
Forks
42
Avg merge
2d 23h
Merged PRs (30d)
1

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from purescript/purescript-quickcheck

All issues in purescript/purescript-quickcheck

Similar issues

More Testing & QA issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.