Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Encode mishandles unrepresentable Shift_JIS characters and FB_PERLQQ

Aperta
  • Qualcosa non va?
#1,629 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
java, perl
Ambito
compilers

Direzione di ricerca

Start with t/pp_07_coder_check_value.t and the loaded t/tests/07_coder_check_value_euc-jp.t regression, then trace the PerlOnJava Encode implementation used by both JVM and interpreter backends. Reproduce the behavior with system Perl, add a focused regression for Shift_JIS replacement and Encode::FB_PERLQQ, and verify both backend results plus the upstream suite.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area:cpan-port area:unicode bug

Summary

Text-CSV-Encoded 0.25's unchanged EUC-JP encoding test passes under system Perl but fails on both PerlOnJava backends. The failure shows that encoding an unrepresentable character to Shift_JIS does not honor Encode's replacement/check behavior: output loses CSV quoting, and Encode::FB_PERLQQ yields U+FFFD instead of the expected \\x{2460} representation.

Reproduction

  • CPAN random tester run: 20261002-182937-4648
  • Distribution: Text-CSV-Encoded 0.25
  • Test: t/pp_07_coder_check_value.t, which loads t/tests/07_coder_check_value_euc-jp.t
  • Environment: macOS arm64, Java 24.0.2, PerlOnJava 5.44.1 (build commit 78521d8d0, 2026-10-02)

The test parses EUC-JP input and combines fields as Shift_JIS. It checks default replacement behavior and then sets encode_check_value(Encode::FB_PERLQQ) before repeating the operation.

System Perl 5.42.2 passes the complete upstream suite (14 files, 255 tests).

PerlOnJava fails both assertions on both backends:

not ok 1
  got:      '...,?'
  expected: '...,"?"'

not ok 2
  got:      '...,\\x{FFFD}'
  expected: '...,"\\x{2460}"'

The result was reproduced with the archived distribution and current PerlOnJava build on both JVM and interpreter backends. The distribution has no XS/C source; its xs_* tests exercise the separate Text::CSV dependency.

Expected behavior

Match system Perl for Shift_JIS encoding of characters that cannot be represented, including the default replacement and FB_PERLQQ check behavior. Preserve the resulting CSV field quoting.

Add a focused project regression for the encoding/check behavior, validate it with system Perl, then verify it on both PerlOnJava backends and rerun the upstream test.

Related issue

Related to #1445, which tracks PerlOnJava Encode compatibility, though that issue covers canonical charset names rather than unrepresentable-character handling.

Lingua principale
Perl
Stelle
64
Fork
7
Merge medio
4h 34m
PR unite (30g)
181

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di fglock/PerlOnJava

Tutte le issue di fglock/PerlOnJava

Issue simili

Altre issue su Perl

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.