[Bug]: Unsupported expression syntax evaluates to None silently instead of raising

Ouverte
  • Quelque chose ne va pas ?
#5,073 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

  • #5206 par @aarjang — fermée sans fusion

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
52/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
python
Domaine
backend, testing

Piste de recherche

Start in src/specify_cli/workflows/expressions.py, especially the parser and evaluation points cited in the report: lines 304, 331, 715, 800, 884, 893, 930-976, and _safe_compare. Read the evaluator tests in tests/test_workflows.py and run the reproductions from the issue for exponent numbers, escaped quotes, chained indexing, and adjacent word operators. Done means each documented case evaluates as expected or raises a clear ValueError for unsupported syntax.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

triage-must-have
Bug Description

When the workflow expression evaluator meets syntax it does not parse, it does not raise. It treats the text as a variable path, which resolves to None (_resolve_dot_path, reached from src/specify_cli/workflows/expressions.py:969). A condition that uses such syntax then quietly gives the wrong result. The following cases were confirmed:

  1. Exponent-form numbers. Numeric literals and numeric strings are parsed with float() only when they contain a ., and otherwise with int() (expressions.py:930-935 for literals, expressions.py:972-976 for _coerce_number). So 1e3 is not read as a number: as a literal it is looked up as a variable and becomes None, and the string "1e3" stays a string in comparisons. Workflow input coercion does accept "1e3" as the number 1000 (src/specify_cli/workflows/engine.py:1449-1456), so the same text means different things in different places.
  2. Escaped quotes in string literals. A string literal is recognised only if its first matching quote is its last character (expressions.py:800), so 'it\'s' is looked up as a variable and becomes None. As a result, inputs.name == 'it\'s' is false for name = "it's".
  3. Chained indexing. The indexed-segment pattern accepts one index only (expressions.py:304), so matrix[0][1] is looked up as the literal key "matrix[0][1]" (expressions.py:331) and returns None.
  4. Word operators next to a quote. and, or and in are only found with a space on both sides (expressions.py:884, expressions.py:893, expressions.py:715). inputs.a=='x'and inputs.b is therefore split only at ==, its right side 'x'and inputs.b is looked up as a variable and becomes None, and the result is false.
Steps to Reproduce
  1. From a fresh clone of spec-kit, run uv sync.
  2. Save the script below as repro.py in the repository root.
  3. Run uv run python -I repro.py.
from specify_cli.workflows.base import StepContext
from specify_cli.workflows.engine import WorkflowEngine
from specify_cli.workflows.expressions import evaluate_condition, evaluate_expression

ctx = StepContext(
    inputs={"n": 5, "s": "1e3", "name": "it's", "a": "x", "b": True},
    steps={"plan": {"output": {"matrix": [["a", "b"], ["c", "d"]]}}},
)
cases = [
    ("1", "{{ 1e3 }}", "value"),
    ("1", "{{ inputs.n < 1e3 }}", "cond"),
    ("1", "{{ inputs.s > 100 }}", "cond"),
    ("2", r"{{ 'it\'s' }}", "value"),
    ("2", r"{{ inputs.name == 'it\'s' }}", "cond"),
    ("3", "{{ steps.plan.output.matrix[0] }}", "value"),
    ("3", "{{ steps.plan.output.matrix[0][1] }}", "value"),
    ("4", "{{ inputs.a == 'x' and inputs.b }}", "cond"),
    ("4", "{{ inputs.a=='x'and inputs.b }}", "cond"),
    ("4", "{{ 'x'in inputs.a }}", "cond"),
]
for num, expr, kind in cases:
    fn = evaluate_expression if kind == "value" else evaluate_condition
    print(f"[{num}] {expr} -> {fn(expr, ctx)!r}")
print("input coercion of '1e3' (type: number):",
      WorkflowEngine._coerce_input("x", "1e3", {"type": "number"}))
Expected Behavior

The module docstring describes the evaluator as "a safe Jinja2 subset" (src/specify_cli/workflows/expressions.py:3), and _evaluate_simple_expression lists "String and numeric literals" as supported (expressions.py:791). For each case, the expression should either evaluate as it would in Jinja2 or raise a ValueError naming the unsupported syntax. Malformed filters already get such a ValueError (docs/reference/workflows.md, "Types are validated, not coerced"). Specifically:

  1. {{ 1e3 }} should be 1000.0, and {{ inputs.n < 1e3 }} should be true for n = 5. {{ inputs.s > 100 }} with s = "1e3" should be true, in line with the _safe_compare docstring (expressions.py:1003-1012) and the input-coercion test tests/test_workflows.py:7092 (("1e3", 1000)).
  2. {{ 'it\'s' }} should be "it's", and the comparison should be true.
  3. {{ steps.plan.output.matrix[0][1] }} should be 'b'. The _resolve_dot_path docstring promises list indexing (expressions.py:312).
  4. {{ inputs.a=='x'and inputs.b }} should be true, like the spaced form. {{ 'x'in inputs.a }} should be true.
Actual Behavior
[1] {{ 1e3 }} -> None
[1] {{ inputs.n < 1e3 }} -> False
[1] {{ inputs.s > 100 }} -> False
[2] {{ 'it\'s' }} -> None
[2] {{ inputs.name == 'it\'s' }} -> False
[3] {{ steps.plan.output.matrix[0] }} -> ['a', 'b']
[3] {{ steps.plan.output.matrix[0][1] }} -> None
[4] {{ inputs.a == 'x' and inputs.b }} -> True
[4] {{ inputs.a=='x'and inputs.b }} -> False
[4] {{ 'x'in inputs.a }} -> False
input coercion of '1e3' (type: number): 1000
Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Not applicable

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

When cases 1, 3 and 4 are used as an if condition, validate_workflow returns no errors. Case 2 is flagged in a condition, but as unbalanced delimiters and quotes rather than as an escaped quote.

Related: #5060 proposes replacing the hand-written expression parser with a grammar so that invalid input raises. It does not mention this case.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.

Langage dominant
Python
Étoiles
149k
Forks
13.4k
Merge moyen
1 j 5 h
PR mergées (30 j)
223

Préparer son environnement

Ouvrir dans Codespaces

Lance le conteneur de développement du projet dans votre navigateur, avec votre propre compte GitHub.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de github/spec-kit

Toutes les issues de github/spec-kit

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.