Hacktoberfest 2026：維護者為十月標記出來的 issue，仍然開放、適合新手。 瀏覽 Hacktoberfest issue

`codecs.StreamReader.read()` silently drops an incomplete character at the end of the stream

未關閉
  • 哪裡不對？
#160,309 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 1 天內回覆

@saiyam-jain-dev 已經在處理了。

開始於 2026年10月25日。

  • #160642 來自 @saiyam-jain-dev —— 未關閉

評估

難度
4/5
預估耗時
3-5 天
新手友好度
56/100
Issue 類型
缺陷
描述清晰度
描述清楚
活躍度
活躍
技術堆疊
python
領域
backend, testing

研究方向

Start with Lib/codecs.py around the StreamReader.read() implementation and the linked expected-failure test StreamReaderTruncationBug.test_read_reports_truncated_input from PR #160155. Compare its end-of-stream behavior with IncrementalDecoder.decode(b'', final=True) and check the reported UTF-8, UTF-16, UTF-32, and escape-codec cases. Done means truncated input raises or uses the selected error handler instead of being silently discarded.

由索引模型根據 Issue 內容生成。

描述

stdlib type-bug

Bug description

StreamReader.read() decodes with the stateless codec function and keeps any undecoded trailing bytes for the next call. When the stream is exhausted there is no next call, so an incomplete final character is neither reported nor replaced; it just disappears:

>>> import codecs, io
>>> reader = codecs.getreader('utf-8')(io.BytesIO(b'ab\xe2'))
>>> reader.read()
'ab'
>>> reader.read()
''

Every other way of decoding the same bytes either raises or applies the error handler:

>>> b'ab\xe2'.decode('utf-8')
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    b'ab\xe2'.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 2: unexpected end of data
>>> io.TextIOWrapper(io.BytesIO(b'ab\xe2'), encoding='utf-8').read()
Traceback (most recent call last):
  ...
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 2: unexpected end of data
>>> codecs.getreader('utf-8')(io.BytesIO(b'ab\xe2'), 'replace').read()
'ab'
>>> b'ab\xe2'.decode('utf-8', 'replace')
'ab�'

So with errors='strict' a truncated file reads as if it were well-formed, and with 'replace' the replacement character that would mark the damage is missing. The same happens with utf-16, utf-32 and the escape codecs (unicode_escape with a trailing backslash); the CJK codecs are not affected, because their stateless decoders raise on an incomplete sequence themselves.

The cause is in read(): self.decode is the codec's stateless function with its default final=False (Lib/codecs.py line 512), and the loop exits on empty newdata without ever telling the decoder that the input is complete; self.bytebuffer keeps the fragment forever. Fixing it would mean a final decode pass once the underlying stream returns nothing, and raising (or applying the handler) if bytes remain, which is what IncrementalDecoder.decode(b'', final=True) does. The incremental decoders did not exist when StreamReader was written, which I assume is how this came about.

I realise codecs.open() is deprecated since 3.14 and StreamReader is on its way out with it, so feel free to say this is not worth touching; but it is silent data loss in the strict mode, which seemed worth having on record.

This is the same mechanism as #56717 (bpo-12508, 2011), which describes it for UTF-8 with errors='replace' and has had no activity since; I am filing it anyway because it applies to every codec with multi-byte characters and the strict-mode data loss is the part that matters, but feel free to fold this into that one.

Found by the Hypothesis tests in #160155, where it is pinned as StreamReaderTruncationBug.test_read_reports_truncated_input under expectedFailure.

CPython versions tested on

CPython main (6e1bd24e4d8)

Operating systems tested on

Linux

主要語言
Python
星號
77.7k
分支
38.3k
平均合併
13 小時 30 分鐘
30 天內合併 PR
708

環境準備

在 Codespaces 中開啟

在瀏覽器裡用你自己的 GitHub 帳號啟動這個專案的開發容器。

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

python/cpython 的其他 Issue

查看 python/cpython 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。