microsoft/Terminal

A few StateMachine unit tests don't fully test what they purport to

Open

#1,372 opened on Jun 21, 2019

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (3,212 forks)batch import
Area-CodeHealthHelp WantedIssue-TaskProduct-Conhost

Repository metrics

Stars
 (35,764 stars)
PR merge metrics
 (Avg merge 27d 19h) (24 merged PRs in 30d)

Description

I'm not sure if this is right place to report this, because it's not a bug in the app itself, but it's an issue I noticed in the unit tests, which I think may be considered a bug.

As a specific example, consider the TestCursorKeysMode test in OutputEngineTest.cpp:

TEST_METHOD(TestCursorKeysMode)
{
    StatefulDispatch* pDispatch = new StatefulDispatch;
    VERIFY_IS_NOT_NULL(pDispatch);
    StateMachine mach(new OutputStateMachineEngine(pDispatch));

    mach.ProcessString(L"\x1b[?1h", 5);
    VERIFY_IS_TRUE(pDispatch->_fCursorKeysMode);

    pDispatch->ClearState();

    mach.ProcessString(L"\x1b[?1l", 5);
    VERIFY_IS_FALSE(pDispatch->_fCursorKeysMode);

    pDispatch->ClearState();
}

The second half of this test is assumedly meant to prove that the given escape sequence will reset the cursor keys mode. However, the _fCursorKeysMode flag is false by default, so you could replace that escape sequence with almost anything, including removing it altogether, and the test would still pass.

There are similar problems with TestCursorBlinking, TestCursorVisibility, and possibly also TestAltBufferSwapping.

When I added a similar test, I got around the problem by explicitly setting the flag I was testing to the opposite of what was expected, but I'm not sure if that is the best approach. Another possibility might be using std::optional<bool> instead of just a bool, so the default value could then be a nullopt, and it wouldn't be possible to pass the test based on the default value alone. The downside of that approach is the VERIFY steps become a little more complicated.

Contributor guide