toMatchDisk_TODO throws error when writing multiple snapshots to disk during parameterized tests, succeeds on follow ups.

Open
#370 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
kotlin
Domain
testing

Research direction

Start with the parameterized Kotlin reproducer using selfie-runner-junit5:2.0.1 and inspect the failure at Validator.kt:613. Compare the first run, when the snapshot file is created, with later runs and with reuse of an existing _TODO; done means multiple snapshots can be written without the “Snapshot was set to multiple values!” error.

Written by the indexing model from the issue text.

Description

bug jvm

I am using selfie-runner-junit5:2.0.1 to verify errors for each test case. I my parameters include a unique test case name to pass into toMatchDisk.

The summarized version:

    @ParameterizedTest(name = "rejects invalid commands: {1}")
    @MethodSource("modifyNodeItemsValidationCases")
    fun `rejects invalid commands`(
        cmd: Command,
        testcaseName: String,
    ) = runTest {
        val thrown = shouldThrow<TraitStandardResponseException> {
            validate(cmd)
        }
        thrown.response.respCase shouldBe RespCase.RESPONSE_BAD_REQUEST
        Selfie.expectSelfie(thrown.message.toString()).toMatchDisk(testcaseName)
    }

When I run this test case, the snapshot file is successfully written:

╔═ rejects invalid commands/Child node not found ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/Empty modify item ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/Empty modify node item ═╗
ModifyNodeItemWithAddition must have either modifyItem, updateExistingTraits, modifyChildElectricalConnectionPoints, or upstreamElectricalConnectionPoint
╔═ rejects invalid commands/No UtilityMeterNode when trying to edit a utility meter node ═╗
No UtilityMeterNode item found for node id 1 in topology
╔═ rejects invalid commands/Node not found ═╗
Provided node id 1 not in topology
╔═ rejects invalid commands/empty modifyDownstreamElectricalConnectionPoint ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/empty simpleElectricalConnectionPoint ═╗
ModifyItem must have a valid item case to modify
╔═ [end of file] ═╗

But the first run fails with this error:

org.opentest4j.AssertionFailedError: Snapshot was set to multiple values!
  first time: ValidatorTest$rejects invalid commands$1.invokeSuspend(Validator.kt:613)
    this time: ValidatorTest$rejects invalid commands$1.invokeSuspend(Validator.kt:613)

Once the file is created things seem to work:

  • Subsequent test runs pass
  • If I modify one or more of the expected value and include //selfieonce the updates are made and tests pass
  • Add one or more new test case with //selfieonce

I can produce the same errors again if the file exists if I ever reuse the _TODO.

Dominant language
Kotlin
Stars
101
Forks
18
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from diffplug/selfie

All issues in diffplug/selfie

Similar issues

More Kotlin issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.