Increase specificity of integration tests

Open
#554 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
50/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Stale
Tech stack
lua
Domain
testing-qa

Research direction

Identify the integration test files containing the listed tests (e.g., testChainingPropagationThroughSwap1, testFirstHoverFrameMatch, frameTricksTest). For each, replace full replay loading with minimal puzzle setups using scheduled garbage and replay inputs. Run the relevant test files to confirm behavior is preserved. Done when all scoped tests are converted and the test suite passes.

Written by the indexing model from the issue text.

Description

Cleanup engine priority: 0 - high

Our current integration tests for engine functionality are very broad in the manner that they load up an entire replay and play it, only pausing at specific moments to verify behaviour.
As the engine changes over time to provide a better gameplay experience, replays used in integration tests may break for reasons unrelated to the actual change in the engine.
For most of these integration tests it should be possible to verify the behaviour by creating a puzzle with extra steps e.g. garbage scheduled to fall at specific times along with replay inputs that mimic the actions in the original replay.
This way the approximately 99% of each replay that aren't relevant for the behaviour verification can be eliminated and integration tests are likely to hold even through engine changes.

List of tests for conversion within the scope of this issue:

  • testChainingPropagationThroughSwap1
  • testFirstHoverFrameMatch
  • testHoverChainOverGarbageClear
  • horizontalSwapIntoHoverTest
  • noInputsInVsIsDrawTest
  • frameTricksTest; should be divided into multiple:
    • stealth
    • insert
    • outsert
    • platform
    • catch
    • grounded catch
    • DAS catch
    • stealth catch
    • grounded stealth catch
    • slide
    • bloop
    • yoschi's dumptruck
    • tornado
    • mato special
  • catchAndSyncTest (only the sync)
  • downStackDropsSandwichedGarbageAllTogether
  • matchMetalAndGarbageClearsAllMetalTest
  • fallingWhileHoverBeginsDoesNotChain
  • StackRollbackReplayTests
    • rollback from death
    • rolling back around attack timings in all discrete configurations
  • simpleTouchTest; all variations mentioned in the comments in separate tests
  • GarbageQueue tests; these should be remade to only feature the GarbageQueue interface instead of Match
  • testHoverInheritanceOverSwapOverGarbageHover (for the hover)
  • testHoverInheritanceOverSwapOverGarbageHover (for shock panel generation; that cannot be easily replicated with the kind of test demanded by this issue; get creative and probably do this last)

The following tests are intentionally unspecific in nature; we cannot really do anything but replace them when implementing breaking changes that affect a wider array of replays:

  • basicEndlessTest
  • basicTimeAttackTest
  • basicVsTest
  • basicVsTest2

The following tests will just become plain obsolete when implementing breaking changes that affect a wider array of replays, usually because they are testing for compatibility with older engine versions:

  • movingBeforeInPositionDisallowedPriorToTouch

Non-maintenance issues that would make this a lot easier to implement:

  • #137
  • #202

High priority as it is blocking several engine changes and bugfixes, including:

  • #663
  • #634
  • #624
Dominant language
Lua
Stars
30
Forks
20
Avg merge
3d 15h
Merged PRs (30d)
17

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from panel-attack/panel-game

All issues in panel-attack/panel-game

Similar issues

More Lua issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.