Document iconv implementation quirks

Open Beginner friendly
#5,646 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
70/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Quiet
Tech stack
php
Domain
documentation

Research direction

Start with the PHP iconv() manual page linked in the issue and compare its current //TRANSLIT and //IGNORE notes with the cited POSIX iconv_open() and iconv() specifications. Update the documentation to cover implementation-dependent suffix support, //NON_IDENTICAL_DISCARD, and implementation-defined charsets, while preserving the page’s existing guidance.

Written by the indexing model from the issue text.

Description

tl;dr document some if the problems I ran into in https://github.com/php/php-src/issues/22413

PHP's iconv() function mostly defers to the system's iconv_open() and iconv() routines. POSIX 2024 covers these:

As you can see, there are three magic suffixes that can be appended to the "to" charset:

  • //IGNORE
  • //TRANSLIT
  • //NON_IDENTICAL_DISCARD

In practice, support for these varies widely, and when they are supported they may not conform to POSIX. For example, musl does not support //IGNORE, and (man 3 iconv) BSD/Solaris replace unknown translations with ? rather than raise an error that would require //TRANSLIT... so they do not have //TRANSLIT.

The details are sordid, and we should not try to capture them in the PHP documentation. Instead, we should simply state that support for any iconv() feature can vary widely depending on your implementation. Right now the PHP iconv docs mention //TRANSLIT and //IGNORE, but say only that //TRANSLIT is unreliable.

  1. We should also mention that //IGNORE is implementation-dependent. PHP has a workaround for iconv implementations (glibc) that return an error upon ignoring an invalid sequence, which helps standardize PHP's behavior, but //IGNORE may not be supported at all (musl), and there is no workaround for that.
  2. //NON_IDENTICAL_DISCARD needs a mention.
  3. The "to" and "from" charsets are also implementation-defined. Musl for example only supports a few, and will fall back to UTF-8 if you pass in something it doesn't understand.
Dominant language
XML
Stars
596
Forks
890
Avg merge
1d 15h
Merged PRs (30d)
55

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from php/doc-en

All issues in php/doc-en

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.