`it_trouble_shooting_hub/asset_retirement_migration` can never pass: verifier sends `archived` to `databases.query`

Open Beginner friendly
#269 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python
Domain
testing-qa

Research direction

Start with tasks/notion/standard/it_trouble_shooting_hub/asset_retirement_migration/verify.py at line 124 and compare the documented databases.query parameters with the live API behavior described in the issue. Check the corresponding easy-suite verifier at tasks/notion/easy/it_trouble_shooting_hub/simple__asset_retirement_migration/verify.py:92. Done means both verifiers can query the database and return a verdict without the validation error.

Written by the indexing model from the issue text.

Description

tasks/notion/standard/it_trouble_shooting_hub/asset_retirement_migration/verify.py:124
passes archived=False to notion.databases.query:

remaining_items = notion.databases.query(
    database_id=inventory_db_id,
    filter=compound_filter,
    archived=False,
).get("results", [])

archived is not among the documented body parameters of
POST /v1/databases/{id}/query, and the endpoint already returns only non-archived
pages ("If no filter is provided, non-archived pages in the database will be
returned"). notion-client==2.4.0 forwards the kwarg anyway (api_endpoints.py
picks archived into the body), and the API rejects the request:

notion_client.errors.APIResponseError: body failed validation:
body.archived should be not present, instead was `false`.

The call is reached whenever an agent gets far enough to have built the retirement
queue, so the task fails for every agent that does the work correctly — the
verifier raises before it can return a verdict. Agents that fail an earlier check
never reach it, so the task looks like a very hard task rather than a broken one.

Verified against the live API (client sends Notion-Version: 2022-06-28, the
oldest, so there is no API version where this is accepted):

WITH    archived=False : APIResponseError: body failed validation: body.archived should be not present, instead was `false`.
WITHOUT archived       : HTTP 200

We hit it on 20 of 25 runs of this task across five different agent scaffolds.

The easy suite's twin has the same line:
tasks/notion/easy/it_trouble_shooting_hub/simple__asset_retirement_migration/verify.py:92.

Fix is to delete the kwarg — the resulting call is what the code intends, since the
default already excludes archived pages. PR follows.

This file has not changed since it was added in #225 (Nov 2025). The Verified pass
in #264 stabilized nine standard Notion verifiers — including
it_trouble_shooting_hub/verification_expired_update, a sibling in the same task
family — but never touched this one, so the defect survived that sweep.

Since standard is now the MCPMark Verified set, any published Verified score that
includes this task carries a forced miss on it.

Dominant language
Python
Stars
461
Forks
48
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from eval-sys/mcpmark

All issues in eval-sys/mcpmark

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.