Expression column with dtype="int" loses precision for integers above `2**53`
维护者通常 1 天内回复
已经有一个关联 PR 被合并了。
- #1099 来自 @v0ropaev —— 已合并
评估
调研方向
该 bug 位于 packages/data-designer-engine/src/data_designer/engine/column_generators/generators/expression.py 中第 123 行处的 _cast_type。阅读该方法以及建议在 ValueError 时回退到 int(value) 的逻辑。Done 表示大整数在 expression 列中保持精度，而像 "3.0" 这样的十进制字符串仍像之前一样转换；使用复现脚本和测试套件进行验证。
由索引模型根据 Issue 内容生成。
描述
Priority Level
Medium (Annoying but has workaround)
Describe the bug
An expression column with dtype="int" can silently change large integers. ExpressionColumnGenerator._cast_type converts the rendered string with int(float(value)), so an integer above 2**53 that a float can't hold exactly gets rounded to the nearest one it can. There is no warning and no dropped row. The column just holds a different number.
This matters when an expression copies or derives an ID column from seed data, since 64-bit snowflake style IDs and nanosecond timestamps are in this range. The seed column keeps the right value and the expression column next to it doesn't.
Code location is packages/data-designer-engine/src/data_designer/engine/column_generators/generators/expression.py, line 123 on current main (207eda6d).
elif self.config.dtype == "int":
return int(float(value))
Steps/Code to reproduce bug
import pandas as pd
import data_designer.config as dd
from data_designer.interface import DataDesigner
seed = pd.DataFrame({"user_id": [9007199254740993, 1234567890123456789]})
builder = dd.DataDesignerConfigBuilder()
builder.with_seed_dataset(dd.DataFrameSeedSource(df=seed))
builder.add_column(dd.ExpressionColumnConfig(name="user_id_copy", expr="{{ user_id }}", dtype="int"))
result = DataDesigner(artifact_path="./artifacts").preview(builder, num_records=2)
df = result.dataset
print(df[["user_id", "user_id_copy"]].to_string())
print(df.dtypes.to_dict())
print((df["user_id"] == df["user_id_copy"]).tolist())
Output with log lines trimmed, on main at 207eda6d with Python 3.14.
user_id user_id_copy
0 9007199254740993 9007199254740992
1 1234567890123456789 1234567890123456768
{'user_id': dtype('int64'), 'user_id_copy': dtype('int64')}
[False, False]
With 2**63 - 1 in the seed it goes a bit further. The value comes back as 2**63, which doesn't fit in int64, so the column ends up as uint64.
Expected behavior
user_id_copy should equal user_id. Jinja renders the exact digits, so the cast to int should keep them. Rendered values like "3.0" from a division should keep working as they do today.
Agent Diagnostic / Prior Investigation
I traced the value through the generator. The string Jinja renders is exact ("9007199254740993"). The digits change in _cast_type. A dtype="str" column over the same seed keeps them. I searched open and closed issues and PRs for precision, large integer, int64, dtype int, cast, ExpressionColumnConfig and expression column and did not find a report of this. #749 and #757 added row drops for cast failures but did not change the int conversion.
Additional context
One option is to try int(value) first and fall back to int(float(value)) only on ValueError, so strings like "3.0" or "1e3" behave as they do now. I haven't run the test suite against that. Happy to send a PR for this once it's triaged.
Checklist
- I reproduced this issue or provided a minimal example
- I searched the docs/issues myself, or had my agent do so
- If I used an agent, I included its diagnostics above
- 主要语言
- Python
- 星标
- 2.4k
- 派生
- 241
- 平均合并
- 1 天 1 小时
- 30 天内合并 PR
- 86
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
NVIDIA-NeMo/DataDesigner 的其他 Issue
-
enhancement triaged
难度 2/5 1-3 小时 新手友好度 72/100
NVIDIA-NeMo/DataDesigner#1110 · 1 条评论 ·
维护者通常 1 天内回复
-
Local seed path containing a single quote breaks the DuckDB query可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 2/5 1-3 小时 新手友好度 67/100
NVIDIA-NeMo/DataDesigner#1102 · 2 条评论 ·
维护者通常 1 天内回复
-
Code validation repeats the first result for subsequent target columns可能已有人在做 @edjson 于 1 天前认领。 未关闭bug triaged
难度 2/5 1-3 小时 新手友好度 77/100
NVIDIA-NeMo/DataDesigner#1092 · 2 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
enforce \from future import annotations` via ruff FA102 rule`可能已有人在做 @chethanuk 于 49 天前认领。 未关闭task
难度 2/5 1-3 小时 新手友好度 82/100
NVIDIA-NeMo/DataDesigner#760 ·
维护者通常 1 天内回复
-
Support image columns and media collection in multi-shard Slurm runs可能已有人在做 @nabinchha 今天认领。 未关闭enhancement
NVIDIA-NeMo/DataDesigner#1114 · 已指派 1 人 ·
维护者通常 1 天内回复
查看 NVIDIA-NeMo/DataDesigner 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
-
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 2 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 82/100
developmentseed/titiler-cmr#278 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 2 天内回复