Quick/Nimble

Test fails: "expected to eventually be true, got <true>"

Open

#360 opened on Nov 2, 2016

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Swift (3,602 stars) (407 forks)batch import
bughelp wantedquestion / needs more detailstricky

Description

Nimble: 5.1.0, xcode: 8.0 (8A218a), using cocoapods, UI Testing bundle, iOS simulator (iPad 2 Air)

I really like Nimble, thanks!

Observed behaviour: Tests sometimes fail with the following message: "expected to eventually be true, got ". When rerunning the same test it often just passes.

Expected behaviour: When expectations gets true using the matcher beTrue() it should not fail.

Steps to reproduce:

  • I haven't been able to consistently reproduce. The same tests sometimes fail with this message and sometimes don't. It happens in different test classes.
  • The code is always of the following form: expect(myFunc()).toEventually(beTrue(), timeout: myTimeout) where myFunc() is in a different helper class.
  • The same happens with the beTruthy() matcher.

Screenshot from Xcode

Contributor guide

Tech stack
swift
Domain
testing
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
SwiftNimbleasynchronous testing
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Investigate the toEventually implementation in Nimble's source code to understand why it sometimes returns true but the matcher still fails. Check for race conditions in the polling logic. Look at the difference between beTrue and beTruthy matchers and see if the issue is in the matcher evaluation or the asynchronous wait mechanism. Consider adding logging to reproduce the issue consistently.