[Bug]: Unsupported expression syntax evaluates to None silently instead of raising

Đang mở
  • Có gì không ổn?
#5,073 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

  • #5206 của @aarjang — đã đóng, không merge

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
52/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
python
Lĩnh vực
backend, testing

Hướng nghiên cứu

Start in src/specify_cli/workflows/expressions.py, especially the parser and evaluation points cited in the report: lines 304, 331, 715, 800, 884, 893, 930-976, and _safe_compare. Read the evaluator tests in tests/test_workflows.py and run the reproductions from the issue for exponent numbers, escaped quotes, chained indexing, and adjacent word operators. Done means each documented case evaluates as expected or raises a clear ValueError for unsupported syntax.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

triage-must-have
Bug Description

When the workflow expression evaluator meets syntax it does not parse, it does not raise. It treats the text as a variable path, which resolves to None (_resolve_dot_path, reached from src/specify_cli/workflows/expressions.py:969). A condition that uses such syntax then quietly gives the wrong result. The following cases were confirmed:

  1. Exponent-form numbers. Numeric literals and numeric strings are parsed with float() only when they contain a ., and otherwise with int() (expressions.py:930-935 for literals, expressions.py:972-976 for _coerce_number). So 1e3 is not read as a number: as a literal it is looked up as a variable and becomes None, and the string "1e3" stays a string in comparisons. Workflow input coercion does accept "1e3" as the number 1000 (src/specify_cli/workflows/engine.py:1449-1456), so the same text means different things in different places.
  2. Escaped quotes in string literals. A string literal is recognised only if its first matching quote is its last character (expressions.py:800), so 'it\'s' is looked up as a variable and becomes None. As a result, inputs.name == 'it\'s' is false for name = "it's".
  3. Chained indexing. The indexed-segment pattern accepts one index only (expressions.py:304), so matrix[0][1] is looked up as the literal key "matrix[0][1]" (expressions.py:331) and returns None.
  4. Word operators next to a quote. and, or and in are only found with a space on both sides (expressions.py:884, expressions.py:893, expressions.py:715). inputs.a=='x'and inputs.b is therefore split only at ==, its right side 'x'and inputs.b is looked up as a variable and becomes None, and the result is false.
Steps to Reproduce
  1. From a fresh clone of spec-kit, run uv sync.
  2. Save the script below as repro.py in the repository root.
  3. Run uv run python -I repro.py.
from specify_cli.workflows.base import StepContext
from specify_cli.workflows.engine import WorkflowEngine
from specify_cli.workflows.expressions import evaluate_condition, evaluate_expression

ctx = StepContext(
    inputs={"n": 5, "s": "1e3", "name": "it's", "a": "x", "b": True},
    steps={"plan": {"output": {"matrix": [["a", "b"], ["c", "d"]]}}},
)
cases = [
    ("1", "{{ 1e3 }}", "value"),
    ("1", "{{ inputs.n < 1e3 }}", "cond"),
    ("1", "{{ inputs.s > 100 }}", "cond"),
    ("2", r"{{ 'it\'s' }}", "value"),
    ("2", r"{{ inputs.name == 'it\'s' }}", "cond"),
    ("3", "{{ steps.plan.output.matrix[0] }}", "value"),
    ("3", "{{ steps.plan.output.matrix[0][1] }}", "value"),
    ("4", "{{ inputs.a == 'x' and inputs.b }}", "cond"),
    ("4", "{{ inputs.a=='x'and inputs.b }}", "cond"),
    ("4", "{{ 'x'in inputs.a }}", "cond"),
]
for num, expr, kind in cases:
    fn = evaluate_expression if kind == "value" else evaluate_condition
    print(f"[{num}] {expr} -> {fn(expr, ctx)!r}")
print("input coercion of '1e3' (type: number):",
      WorkflowEngine._coerce_input("x", "1e3", {"type": "number"}))
Expected Behavior

The module docstring describes the evaluator as "a safe Jinja2 subset" (src/specify_cli/workflows/expressions.py:3), and _evaluate_simple_expression lists "String and numeric literals" as supported (expressions.py:791). For each case, the expression should either evaluate as it would in Jinja2 or raise a ValueError naming the unsupported syntax. Malformed filters already get such a ValueError (docs/reference/workflows.md, "Types are validated, not coerced"). Specifically:

  1. {{ 1e3 }} should be 1000.0, and {{ inputs.n < 1e3 }} should be true for n = 5. {{ inputs.s > 100 }} with s = "1e3" should be true, in line with the _safe_compare docstring (expressions.py:1003-1012) and the input-coercion test tests/test_workflows.py:7092 (("1e3", 1000)).
  2. {{ 'it\'s' }} should be "it's", and the comparison should be true.
  3. {{ steps.plan.output.matrix[0][1] }} should be 'b'. The _resolve_dot_path docstring promises list indexing (expressions.py:312).
  4. {{ inputs.a=='x'and inputs.b }} should be true, like the spaced form. {{ 'x'in inputs.a }} should be true.
Actual Behavior
[1] {{ 1e3 }} -> None
[1] {{ inputs.n < 1e3 }} -> False
[1] {{ inputs.s > 100 }} -> False
[2] {{ 'it\'s' }} -> None
[2] {{ inputs.name == 'it\'s' }} -> False
[3] {{ steps.plan.output.matrix[0] }} -> ['a', 'b']
[3] {{ steps.plan.output.matrix[0][1] }} -> None
[4] {{ inputs.a == 'x' and inputs.b }} -> True
[4] {{ inputs.a=='x'and inputs.b }} -> False
[4] {{ 'x'in inputs.a }} -> False
input coercion of '1e3' (type: number): 1000
Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Not applicable

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

When cases 1, 3 and 4 are used as an if condition, validate_workflow returns no errors. Case 2 is flagged in a condition, but as unbalanced delimiters and quotes rather than as an escaped quote.

Related: #5060 proposes replacing the hand-written expression parser with a grammar so that invalid input raises. It does not mention this case.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.

Ngôn ngữ chính
Python
Star
149k
Fork
13.4k
Merge trung bình
1 ngày 5 giờ
Pull request đã merge (30 ngày)
223

Chuẩn bị môi trường

Mở trong Codespaces

Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của github/spec-kit

Tất cả issue của github/spec-kit

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.