Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

conformance(runner): reject boolean/float level, invalid fmt, and malformed caller options at run() entry

Open
  • Is something wrong?
#626 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@drk-ram is already working on this.

Since Oct 25, 2026.

  • #631 by @drk-ram — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
testing

Research direction

Start with conformance/src/trace_tests/runner.py, especially run() and _run_levels() around lines 128–185, and review the linked pull request #631 to see the work already underway. Check how the entry points validate their arguments; done means invalid levels and caller options are rejected at both entry points.

Written by the indexing model from the issue text.

Description

needs-review:MEDIUM
Summary

run() and _run_levels() in conformance/src/trace_tests/runner.py (lines 128-185) have entry-validation gaps:

  1. Boolean and integral-float level accepted in run() (conformance/src/trace_tests/runner.py lines 142-143):

    • _run_levels() checks any(type(level) is not int for level in attempted), but run() only checks if level not in _LEVEL_MODULES:.
    • Because True == 1, False == 0, and 1.0 == 1 in Python dict lookup, run(record, fmt, True), run(record, fmt, False), and run(record, fmt, 1.0) silently execute Level 1 / Level 0 instead of raising ValueError.

  2. Unvalidated caller parameters (record, fmt, max_age_seconds, expected_nonce, receipt, entry, policy_resolver) at run() and _run_levels() entry:

    • Passing a non-dict record or an unknown fmt (anything other than "trace" or "cmcp-runtime") is not rejected at entry (extract_trace falls through to returning record for any fmt != "cmcp-runtime").
    • Passing max_age_seconds=True, max_age_seconds=0, max_age_seconds=-1, a non-string expected_nonce, a non-dict receipt/entry, or a non-callable policy_resolver is not rejected at run() / _run_levels() entry.
Proposed Fix
  • Validate type(level) is int and level in _LEVEL_MODULES in run().
  • Validate isinstance(record, dict), fmt in ("trace", "cmcp-runtime"), type(max_age_seconds) is int and max_age_seconds >= 1, expected_nonce is None or isinstance(expected_nonce, str), receipt is None or isinstance(receipt, dict), entry is None or isinstance(entry, dict), and policy_resolver is None or callable(policy_resolver) at entry in both run() and _run_levels().
Dominant language
Python
Stars
37
Forks
33
Avg merge
1d 7h
Merged PRs (30d)
111

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from agentrust-io/trace-spec

All issues in agentrust-io/trace-spec

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.