Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Clarify: Test Object `name` and `suite`

Open
  • Is something wrong?
#64 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
documentation, testing-qa

Research direction

Start with sections 9.4 (name) and 9.9 (suite), then compare their wording with the Python unittest example in the issue. Clarify how the method name, test title, and suite hierarchy should be represented, including whether the title belongs in the standard fields or an extra field; done when the specification gives an unambiguous recommendation.

Written by the indexing model from the issue text.

Description

Please clarify how to document a tests:

  • title
  • name
  • suite hierarchy
Current State of the Spec

Under section "9.4. name" the spec says:

The name or title of the test case.

Under section "9.9. suite" the spec says:

An ordered list of suite or grouping names to which this test belongs, ordered from the top-level suite to the immediate parent of the test.

Situation for xUnit (i.e. Python unittest)

Best explained with an example:

File tests/test_ping.py:

class TestPingTestCase(unittest.TestCase):
    def test_icmp_ping(self):
        """Test Reachability With ICMP Ping
        """
        ... # test code omitted

I would like to document:

  • the suite the test belongs to: ["tests", "test_ping", "TestPingTestCase"]
  • the tests method name: "test_icmp_ping"
  • the tests title (from the so called doc-string): "Test Reachability With ICMP Ping"
  1. According to "9.9 suite", the test methods name does not belong to suite:

    to the immediate parent of the test

  2. Now is left "9.4 name": To ensure that the test can be clearly identified, I need to choose as name the test methods name ("test_icmp_ping"). Also the test method always exists.
  3. The title of the test ("Test Reachability With ICMP Ping") therefore can't be taken into account. Documenting a test is optional anyway and could be documented within e.g. results.tests[].extra.<NAMESPACE>.title.

Resulting in:

  • set name to "test_icmp_ping"
  • set suite to ["tests", "test_ping", "TestPingTestCase"]
  • omit the tests title or e.g. set results.tests[].extra.<NAMESPACE>.title to "Test Reachability With ICMP Ping"

Evaluated and Rejected Alternatives:

  1. Add the tests method name to the "suite" like this: ["tests", "test_ping", "TestPingTestCase", "test_icmp_ping"] and document the optional title under "name". This contradicts the spec for suite stating the "up to the immediate parent" and name being mandatory. --> No go
  2. Keep suite "up to the immediate parent" (["tests", "test_ping", "TestPingTestCase"]) and set name to "test_icmp_ping: Test Reachability With ICMP Ping" to include both the tests method name and title (if available). This potentially could put a burden to the consumer being able to split the method name from the optional title. --> Bad

Would you also recommend to omit the tests title?

Dominant language
No language data
Stars
97
Forks
4
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ctrf-io/ctrf

All issues in ctrf-io/ctrf

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.