socket.recv() and recvfrom() with MSG_TRUNC raise ValueError and lose the datagram since 3.15.1
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 43/100
- Issue 類型
- 缺陷
- 描述清晰度
- 基本清楚
- 活躍度
- 活躍
- 技術堆疊
- python
- 領域
- backend, networking
研究方向
Start by tracing socket.recv() and recvfrom() through sock_recv() and sock_recvfrom() to PyBytesWriter_FinishWithSize(), then inspect the change from gh-157226 and the existing behavior discussed in gh-69121. Reproduce the reported case on Linux and check how recvmsg() handles truncated datagrams. Done means defining and verifying behavior that avoids raising after consuming the datagram and avoids returning invalid padding bytes.
由索引模型根據 Issue 內容生成。
描述
Bug report
Bug description:
Since 3.15.1, socket.recv(bufsize, MSG_TRUNC) and socket.recvfrom(bufsize, MSG_TRUNC) raise ValueError: size larger than allocated size when the datagram is longer than bufsize. Without MSG_PEEK, the datagram is already consumed by the time the exception is raised, so the data is lost:
import socket
a, b = socket.socketpair(socket.AF_UNIX, socket.SOCK_DGRAM)
a.send(b"abcdefgh")
a.send(b"second")
try:
print(b.recv(2, socket.MSG_TRUNC))
except ValueError as e:
print(e)
try:
print(b.recvfrom(2, socket.MSG_TRUNC))
except ValueError as e:
print(e)
b.setblocking(False)
b.recv(100) # BlockingIOError: both datagrams are gone
3.15.1:
size larger than allocated size
size larger than allocated size
3.15.0 and 3.14 (the long-standing behavior, see gh-69121):
b'ab\x00\x00\x00\x00\x00\x00'
(b'se\x00\x00\x00\x00', None)
With MSG_TRUNC, Linux recv(2) returns the real length of the datagram even if it was longer than the buffer. sock_recv() and sock_recvfrom() pass that length to PyBytesWriter_FinishWithSize(), which, after c38a347e2c3199d7d3f6a9eefbf46df52caf725d (backport of gh-157226 to 3.15), rejects a size larger than the allocated one. The same code is in main.
The old behavior was a bug too (gh-69121): the returned object was grown to the real datagram length, with only the first bufsize bytes being valid. Still, it is a behavior change in a bugfix release that breaks working code. For example, CRIU's pycriu used len(s.recv(1, socket.MSG_TRUNC | socket.MSG_PEEK)) to find out the message size (fixed in checkpoint-restore/criu#3268 by using recv_into(), which returns the value from recv(2) as is).
Raising after the data is consumed seems to be the worst option. Returning the min(outlen, bufsize) bytes actually received (as suggested in gh-69121, and similar to what recvmsg() does) would avoid both the data loss and the garbage bytes.
CPython versions tested on:
3.14, 3.15
Operating systems tested on:
Linux
Linked PRs
- gh-160677
- 主要語言
- Python
- 星號
- 77.7k
- 分支
- 38.3k
- 平均合併
- 14 小時 29 分鐘
- 30 天內合併 PR
- 699
環境準備
在瀏覽器裡用你自己的 GitHub 帳號啟動這個專案的開發容器。
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
python/cpython 的其他 Issue
-
topic-socket type-bug
難度 2/5 1-3 小時 新手友好度 72/100
python/cpython#160561 · 7 則留言 ·
維護者通常 1 天內回覆
-
tests topic-profiling type-bug
難度 2/5 1-3 小時 新手友好度 78/100
python/cpython#160334 · 1 則留言 ·
維護者通常 1 天內回覆
-
`iso2022_jp_3` encodes U+9B1D but decodes the result as U+9B1C可能已有人在做 @kinqsradio 今天認領。 未關閉extension-modules type-bug
難度 2/5 1-3 小時 新手友好度 84/100
維護者通常 1 天內回覆
-
interpreter-core type-feature
難度 1/5 1 小時以內 新手友好度 78/100
python/cpython#160109 · 1 則留言 ·
維護者通常 1 天內回覆
-
extension-modules type-bug
難度 2/5 1-3 小時 新手友好度 72/100
python/cpython#160001 · 3 則留言 ·
維護者通常 1 天內回覆
相似的 Issue
-
flaky-watch
難度 2/5 1-3 小時 新手友好度 88/100
btclib-org/btclib#2839 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 65/100
nion-software/nionswift#2018 ·
維護者通常 1 天內回覆
-
Deleting a member fails when another member's share transfer points to them可能已有人在做 @Beerlesklopfer 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
FoodCoopX/wirgarten-tapir#1336 ·
維護者通常 3 天內回覆
-
pydanty:is-working
難度 2/5 1-3 小時 新手友好度 71/100
pydantic/pydantic-ai#11467 ·
維護者通常 1 天內回覆
-
automated issue linkchecker report
難度 2/5 1-3 小時 新手友好度 66/100
QuantEcon/lecture-python.myst#1098 ·
維護者通常 1 天內回覆