Avoid using the terms "latin1" and "iso-8859-1" for isomorphic encoding/decoding

Open
#55 16 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript
Domain
api

Research direction

Start by inspecting the public latin1toString and latin1fromString APIs and searching the repository for their definitions and usages. Read the linked encoding discussions and the issue thread before proposing a breaking rename; done means the project has decided whether to replace the overloaded terminology consistently across its public API.

Written by the indexing model from the issue text.

Description

https://github.com/ExodusOSS/bytes/commit/09878c7cfd7e3fdfebc73a7b3ce45e6d494bac4f made me aware that the functions latin1toString, latin1fromString propagate the confusion between "latin1" and "isomorphic" that we see in a lot of the JavaScript ecosystem, and have tried to help combat in https://github.com/whatwg/encoding/commit/36fb4e7a5582873b3700a5a72912a8e91163a092.

In short, the "latin1" encoding specified in the ISO-8859-1 spec does not provide any encodings for the bytes 0x00 to 0x1F or 0x7F to 0x9F. So a proper latin1 decoder would never return those bytes, and a proper latin1 decoder would throw when given those bytes.

In practice, nobody does this, and we have either:

  • Libraries following the windows-1252 mapping (TextEncoder/TextDecoder, the entire web platform, Node.js's modern standard library);
  • or libraries following the isomorphic decoding / encoding (a lot of C++ code, Node.js's old Buffer API).

This creates a lot of confusion when people expect one of these interpretations and get the other.

My strong suggestion is to never mention the terms latin1 or iso-8859-1 in public APIs, since they mean windows-1252 for people who read standards and mean something else (usually isomorphic encoding) for people who are coming from certain C++ codebases. (I think V8 is the original source of the confusion, at least in the Node.js ecosystem.) Instead, use the standard and non-overloaded term "isomorphic".

I realize this is a breaking change and might not be one you want to take on, but I thought I should file it, in the interest of making this the best encoding/decoding library for JS.

Dominant language
JavaScript
Stars
18
Forks
7
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ExodusOSS/bytes

All issues in ExodusOSS/bytes

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.