bulkcopy() silently truncates fractional seconds in string datetime values to microseconds
维护者通常 2 天内回复
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 48/100
调研方向
Start by tracing how Cursor.bulkcopy() parses string values into DATETIME2 and DATETIMEOFFSET, then compare that path with execute()/executemany() and the Arrow timestamp path. Reproduce with the supplied seven-digit fractional value and trailing-Z example, checking behavior on Python 3.10 and 3.11. Done means bulkcopy preserves all seven digits like executemany and accepts server-valid input, or rejects precision loss explicitly.
由索引模型根据 Issue 内容生成。
描述
Summary
When a string is bound to a DATETIME2 column through Cursor.bulkcopy(), the Rust core parses it with Python's datetime.fromisoformat before sending it. datetime only holds microseconds, so a seven-digit fraction is truncated to six with no warning. The same string through execute()/executemany() is sent to the server as text and SQL Server stores all seven digits.
On Python 3.10 the same parser also rejects a trailing Z, which the server accepts. From 3.11 that case works. This is probably fine since python 3.10 is EOL in a few days, but I wanted to raise since your docs explicitly say you need >=3.10 for bulkcopy.
To reproduce
import uuid
import mssql_python
conn = mssql_python.connect(CONN_STR, autocommit=True)
cur = conn.cursor()
table = f"dbo.Repro_{uuid.uuid4().hex[:8]}"
cur.execute(f"CREATE TABLE {table} (lbl VARCHAR(20), dt DATETIME2)")
value = "2024-07-01 12:00:00.1234567"
cur.executemany(f"INSERT INTO {table} VALUES (?, ?)", [("executemany", value)])
cur.bulkcopy(table, [("bulkcopy", value)])
cur.execute(f"SELECT lbl, CONVERT(varchar(27), dt, 121) FROM {table}")
print(cur.fetchall())
cur.execute(f"DROP TABLE {table}")
Observed:
Python 3.11:
[('executemany', '2024-07-01 12:00:00.1234567'),
('bulkcopy', '2024-07-01 12:00:00.1234560')]
Python 3.10, where fromisoformat also rejects Z:
cur.bulkcopy(table, [("bulkcopy", "2024-07-01T12:00:00Z")])
ValueError: Invalid datetime format '2024-07-01T12:00:00Z'. Expected ISO format like
'2024-01-15 09:30:00' or '2024-01-15T09:30:00': ValueError: Invalid isoformat string: ...
Note:
bulkcopy_arrow() with a timestamp[ns] column stores all seven digits, into both DATETIME2 and DATETIMEOFFSET:
import pyarrow as pa
ns = pa.array([1719835200_123456700], pa.timestamp("ns")) # 2024-07-01 12:00:00.1234567
cur.bulkcopy_arrow(table, pa.table({"lbl": ["arrow"], "dt": ns}))
# -> '2024-07-01 12:00:00.1234567'
Expected behavior
bulkcopy() stores the same value as executemany() for the same input string. In particular, a DATETIME2(7) column receives all seven digits, and inputs that SQL Server accepts are not rejected on the client.
Suggested fixes, in order of preference
- Parse ISO 8601 strings in Rust directly into the TDS DATETIME2/DATETIMEOFFSET representation, which has 100 ns resolution, the same way the Arrow path already consumes nanosecond buffers. This also removes the dependence on the running interpreter's fromisoformat behaviour.
- Alternatively, send string values for datetime columns to the server as text and let it convert, matching the execute path.
- At minimum, raise when a fraction has more digits than can be represented instead of truncating. Silent data loss is the worst of the three outcomes.
- 主要语言
- Python
- 星标
- 481
- 派生
- 62
- 平均合并
- 2 天 11 小时
- 30 天内合并 PR
- 12
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
microsoft/mssql-python 的其他 Issue
-
Make azure-identity an optional dependency可能已有人在做 @jahnvi480 于 3 天前认领。 未关闭area: connectivity-auth dependencies enhancement triage needed
难度 4/5 3-5 天 新手友好度 48/100
microsoft/mssql-python#862 · 3 条评论 · 已指派 1 人 ·
维护者通常 2 天内回复
-
triage needed
难度 5/5 一周以上 新手友好度 38/100
microsoft/mssql-python#850 · 2 条评论 · 1 个 reaction ·
维护者通常 2 天内回复
-
bulkcopy() & bulkcopy_arrow() fail with Schannel AcquireCredentialsHandleW error 0x8009030D (SEC_E_UNKNOWN_CREDENTIALS)可能已有人在做 @jahnvi480 于 11 天前认领。 未关闭area: bulk-copy bug inADO triage done
难度 4/5 3-5 天 新手友好度 20/100
microsoft/mssql-python#848 · 5 条评论 · 已指派 1 人 ·
维护者通常 2 天内回复
-
macOS: encrypted connections fail after Homebrew moved the `openssl` alias to `openssl@4`可能重新可做 @bewithgaurav 于 31 天前认领,目前没有进行中的 PR。 未关闭area: packaging-platform bug triage needed
难度 4/5 3-5 天 新手友好度 55/100
microsoft/mssql-python#826 · 2 条评论 · 4 个 reaction · 已指派 1 人 ·
维护者通常 2 天内回复
-
Announcement: mssql-python is coming to Django through mssql-django 2.0可能重新可做 @bewithgaurav 于 58 天前认领,目前没有进行中的 PR。 未关闭announcement
microsoft/mssql-python#748 · 已指派 1 人 ·
维护者通常 2 天内回复
查看 microsoft/mssql-python 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
namespace operations
难度 1/5 1 小时以内 新手友好度 65/100
EclipseFdn/open-vsx.org#15057 ·
维护者通常 1 天内回复
-
Mission statement: why this is free, in the README, Why This Exists and the wiki Home可能已有人在做 @ChiefGyk3D 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
ChiefGyk3D/Clouds-Against-Humanity#348 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 74/100
vllm-project/vime#508 ·
维护者通常 4 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复