[Bug]: Unsupported expression syntax evaluates to None silently instead of raising

Aperta
  • Qualcosa non va?
#5,073 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

  • #5206 di @aarjang — chiusa senza merge

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, testing

Direzione di ricerca

Start in src/specify_cli/workflows/expressions.py, especially the parser and evaluation points cited in the report: lines 304, 331, 715, 800, 884, 893, 930-976, and _safe_compare. Read the evaluator tests in tests/test_workflows.py and run the reproductions from the issue for exponent numbers, escaped quotes, chained indexing, and adjacent word operators. Done means each documented case evaluates as expected or raises a clear ValueError for unsupported syntax.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

triage-must-have
Bug Description

When the workflow expression evaluator meets syntax it does not parse, it does not raise. It treats the text as a variable path, which resolves to None (_resolve_dot_path, reached from src/specify_cli/workflows/expressions.py:969). A condition that uses such syntax then quietly gives the wrong result. The following cases were confirmed:

  1. Exponent-form numbers. Numeric literals and numeric strings are parsed with float() only when they contain a ., and otherwise with int() (expressions.py:930-935 for literals, expressions.py:972-976 for _coerce_number). So 1e3 is not read as a number: as a literal it is looked up as a variable and becomes None, and the string "1e3" stays a string in comparisons. Workflow input coercion does accept "1e3" as the number 1000 (src/specify_cli/workflows/engine.py:1449-1456), so the same text means different things in different places.
  2. Escaped quotes in string literals. A string literal is recognised only if its first matching quote is its last character (expressions.py:800), so 'it\'s' is looked up as a variable and becomes None. As a result, inputs.name == 'it\'s' is false for name = "it's".
  3. Chained indexing. The indexed-segment pattern accepts one index only (expressions.py:304), so matrix[0][1] is looked up as the literal key "matrix[0][1]" (expressions.py:331) and returns None.
  4. Word operators next to a quote. and, or and in are only found with a space on both sides (expressions.py:884, expressions.py:893, expressions.py:715). inputs.a=='x'and inputs.b is therefore split only at ==, its right side 'x'and inputs.b is looked up as a variable and becomes None, and the result is false.
Steps to Reproduce
  1. From a fresh clone of spec-kit, run uv sync.
  2. Save the script below as repro.py in the repository root.
  3. Run uv run python -I repro.py.
from specify_cli.workflows.base import StepContext
from specify_cli.workflows.engine import WorkflowEngine
from specify_cli.workflows.expressions import evaluate_condition, evaluate_expression

ctx = StepContext(
    inputs={"n": 5, "s": "1e3", "name": "it's", "a": "x", "b": True},
    steps={"plan": {"output": {"matrix": [["a", "b"], ["c", "d"]]}}},
)
cases = [
    ("1", "{{ 1e3 }}", "value"),
    ("1", "{{ inputs.n < 1e3 }}", "cond"),
    ("1", "{{ inputs.s > 100 }}", "cond"),
    ("2", r"{{ 'it\'s' }}", "value"),
    ("2", r"{{ inputs.name == 'it\'s' }}", "cond"),
    ("3", "{{ steps.plan.output.matrix[0] }}", "value"),
    ("3", "{{ steps.plan.output.matrix[0][1] }}", "value"),
    ("4", "{{ inputs.a == 'x' and inputs.b }}", "cond"),
    ("4", "{{ inputs.a=='x'and inputs.b }}", "cond"),
    ("4", "{{ 'x'in inputs.a }}", "cond"),
]
for num, expr, kind in cases:
    fn = evaluate_expression if kind == "value" else evaluate_condition
    print(f"[{num}] {expr} -> {fn(expr, ctx)!r}")
print("input coercion of '1e3' (type: number):",
      WorkflowEngine._coerce_input("x", "1e3", {"type": "number"}))
Expected Behavior

The module docstring describes the evaluator as "a safe Jinja2 subset" (src/specify_cli/workflows/expressions.py:3), and _evaluate_simple_expression lists "String and numeric literals" as supported (expressions.py:791). For each case, the expression should either evaluate as it would in Jinja2 or raise a ValueError naming the unsupported syntax. Malformed filters already get such a ValueError (docs/reference/workflows.md, "Types are validated, not coerced"). Specifically:

  1. {{ 1e3 }} should be 1000.0, and {{ inputs.n < 1e3 }} should be true for n = 5. {{ inputs.s > 100 }} with s = "1e3" should be true, in line with the _safe_compare docstring (expressions.py:1003-1012) and the input-coercion test tests/test_workflows.py:7092 (("1e3", 1000)).
  2. {{ 'it\'s' }} should be "it's", and the comparison should be true.
  3. {{ steps.plan.output.matrix[0][1] }} should be 'b'. The _resolve_dot_path docstring promises list indexing (expressions.py:312).
  4. {{ inputs.a=='x'and inputs.b }} should be true, like the spaced form. {{ 'x'in inputs.a }} should be true.
Actual Behavior
[1] {{ 1e3 }} -> None
[1] {{ inputs.n < 1e3 }} -> False
[1] {{ inputs.s > 100 }} -> False
[2] {{ 'it\'s' }} -> None
[2] {{ inputs.name == 'it\'s' }} -> False
[3] {{ steps.plan.output.matrix[0] }} -> ['a', 'b']
[3] {{ steps.plan.output.matrix[0][1] }} -> None
[4] {{ inputs.a == 'x' and inputs.b }} -> True
[4] {{ inputs.a=='x'and inputs.b }} -> False
[4] {{ 'x'in inputs.a }} -> False
input coercion of '1e3' (type: number): 1000
Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Not applicable

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

When cases 1, 3 and 4 are used as an if condition, validate_workflow returns no errors. Case 2 is flagged in a condition, but as unbalanced delimiters and quotes rather than as an escaped quote.

Related: #5060 proposes replacing the hand-written expression parser with a grammar so that invalid input raises. It does not mention this case.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.

Lingua principale
Python
Stelle
149k
Fork
13.4k
Merge medio
1g 5h
PR unite (30g)
223

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di github/spec-kit

Tutte le issue di github/spec-kit

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.