The punycode incremental encoder and decoder transform each chunk separately
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
- #160599 來自 @OsmnvAslan —— 已關閉，未合併
評估
- 難度
- 3/5
- 預估耗時
- 1-2 天
- 新手友好度
- 72/100
- Issue 類型
- 缺陷
- 描述清晰度
- 描述清楚
- 活躍度
- 活躍
- 技術堆疊
- python
研究方向
Read Lib/encodings/punycode.py around lines 226–234 and the buffered incremental encoder and decoder base classes. Check how idna buffers input and how test_codecs.py marks punycode as broken in stateful mode. Implement buffering until final for both directions, then verify chunked input produces the same result as the whole-string transform.
由索引模型根據 Issue 內容生成。
描述
Bug description
encodings.punycode registers an IncrementalEncoder and IncrementalDecoder, but they just run the whole-string transform on every chunk they are given, so anything fed in more than one piece comes out wrong:
>>> import codecs
>>> 'ab'.encode('punycode')
b'ab-'
>>> encoder = codecs.getincrementalencoder('punycode')()
>>> encoder.encode('a') + encoder.encode('b', final=True)
b'a-b-'
>>> b'a-b-'.decode('punycode')
'a-b'
Punycode encodes the positions of the non-ASCII characters relative to the whole label, so a chunk cannot be encoded without the rest of the string; the same goes for decoding. The classes are two lines each, Lib/encodings/punycode.py lines 226-234, and ignore final altogether. This is half known: test_codecs.py lists punycode in broken_unicode_with_stateful ("encodings that don't work in stateful mode") and skips the chunked part of test_basics for it, but nothing says so in the docs, and the classes are still registered and handed out by codecs.getincrementalencoder(), codecs.iterencode() and StreamWriter.
I think the right fix is to buffer until final (the BufferedIncrementalEncoder and BufferedIncrementalDecoder base classes make that a few lines: _buffer_encode returns (b'', 0) until final), since that keeps codecs.iterencode, StreamWriter and friends working for anyone who hands them the whole label at once, which is presumably every current user. The idna codec already does something like it, buffering up to each ..
Related: #64331 (many incremental codecs do not handle fragmented data) mentions punycode in passing, in a list taken from that same test_codecs.py entry, but does not go into it.
Found by the Hypothesis tests in #160155, where it is pinned as PunycodeIncrementalBug.test_chunked_encode under expectedFailure; the chunked tests there leave punycode out otherwise.
CPython versions tested on
CPython main (6e1bd24e4d8)
Operating systems tested on
Linux
- 主要語言
- Python
- 星號
- 77.7k
- 分支
- 38.3k
- 平均合併
- 11 小時 58 分鐘
- 30 天內合併 PR
- 672
環境準備
在瀏覽器裡用你自己的 GitHub 帳號啟動這個專案的開發容器。
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
python/cpython 的其他 Issue
-
topic-socket type-bug
難度 2/5 1-3 小時 新手友好度 72/100
python/cpython#160561 · 7 則留言 ·
維護者通常 1 天內回覆
-
tests topic-profiling type-bug
難度 2/5 1-3 小時 新手友好度 78/100
python/cpython#160334 · 1 則留言 ·
維護者通常 1 天內回覆
-
extension-modules type-bug
難度 2/5 1-3 小時 新手友好度 84/100
維護者通常 1 天內回覆
-
interpreter-core type-feature
難度 1/5 1 小時以內 新手友好度 78/100
python/cpython#160109 · 1 則留言 ·
維護者通常 1 天內回覆
-
extension-modules type-bug
難度 2/5 1-3 小時 新手友好度 72/100
python/cpython#160001 · 3 則留言 ·
維護者通常 1 天內回覆
相似的 Issue
-
bug P2 rust
難度 2/5 1-3 小時 新手友好度 82/100
維護者通常 1 天內回覆
-
enhancement
難度 2/5 1-3 小時 新手友好度 72/100
jscott3201/rusty-bacnet#2085 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 84/100
evarness-ai/iris-harness#401 ·
-
難度 2/5 1-3 小時 新手友好度 76/100
sistemicorp/scripts#62 ·
-
area/mac help wanted needs-qa
難度 1/5 1 小時以內 新手友好度 76/100