Recipe call arguments that name a data-driven column are not expanded, and the result depends on map order
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 58/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- go
- Domain
- testing-qa
Research direction
Start in internal/runner/executor.go at runRecipeCall and resolve, then read the Data-driven page mentioned in the issue. Reproduce the issue with an Executor unit test using row values in e.vars and recipe arguments containing <email> and <password>. Done when recipe arguments are either expanded consistently before binding or the lint and documentation rule is made explicit, with behavior independent of map order.
Written by the indexing model from the issue text.
Description
What happens
A recipe called from a data-driven test cannot reliably receive the row's columns as call arguments.
recipe "sign in as" (email, password)
type "<email>" into #email
type "<password>" into #password
test "role-based access"
sign in as "<email>" with "<password>"
with examples:
email password
"[email protected]" "admin123"
runRecipeCall binds each argument's text as it is (internal/runner/executor.go, the loop that calls setVar(param, args[i], ...)), so the parameter email is set to the text <email>. That replaces the row's email, and resolve then finds a variable whose value is its own placeholder and makes no progress: the step types <email> literally.
With parameter names that differ from the columns (recipe "sign in as" (address, secret)), the call binds address to the text <email>. resolve makes one pass over the variables in map order, so whether <address> becomes <email> and then the row's value depends on the order the pass visits address and email.
Measured
In a unit test on Executor (newTestExecutor, the row values set in e.vars, runRecipeCall with the arguments "<email>" and "<password>", then e.resolve("type <address> then <secret>")), 300 runs gave four different results:
type <email> then <password>: 179type [email protected] then <password>: 51type <email> then rowpw: 37type [email protected] then rowpw: 33
With the parameters named like the columns, the result is type <email> every time.
Expected
Either the arguments of a recipe call are expanded (against the row and the other variables) before they are bound, or probe lint and the docs say that a column must not be passed as an argument. The Data-driven page now says the latter, and shows a recipe without parameters that reads the columns directly; that form does not depend on the order.
Found while checking the recipe "log in as" example, which also had a keyword problem (fixed separately).
- Dominant language
- Go
- Stars
- 6
- Forks
- 3
- Avg merge
- 6h 28m
- Merged PRs (30d)
- 362
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from AlphaWaveSystems/flutter-probe
-
agent: is checked on a wrapper holding several checkables answers from the first one in tree orderOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
AlphaWaveSystems/flutter-probe#865 ·
Maintainers usually reply within 1 day
-
area/cli bug
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
AlphaWaveSystems/flutter-probe#424 ·
Maintainers usually reply within 1 day
-
area/docs documentation
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
AlphaWaveSystems/flutter-probe#409 ·
Maintainers usually reply within 1 day
-
Dial timeout error should say how long it waited and name the timeout settingPossibly taken @jay23606 claimed this 2 days ago. Openarea/cli bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
AlphaWaveSystems/flutter-probe#408 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
AlphaWaveSystems/flutter-probe#888 ·
Maintainers usually reply within 1 day
All issues in AlphaWaveSystems/flutter-probe
Similar issues
-
agent-review-finding chore
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
jordansmall/spindrift#7254 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pg-sharding/spqr#3275 ·
Maintainers usually reply within 1 day
-
create_or_update_file rejects explicitly empty file contentPossibly taken @Elon-Jiang claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
github/github-mcp-server#3883 ·
Maintainers usually reply within 4 days
-
feature
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
databricks/terraform-provider-databricks#6210 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
IBM-Cloud/terraform-provider-ibm#7115 ·
Maintainers usually reply within 2 days