MAINT Extract scenario-history attempt-to-work-unit matching
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Active
- Domain
- backend, databases, testing-qa
Research direction
Wait for #2766 and read its established query boundary before starting. Then inspect pyrit/memory/memory_interface.py at _build_scenario_history_unit_statement, its subqueries and expressions, and the aggregate builder; run the scenario-history and scenario-run-service tests. Done means the matching logic is extracted without changing result columns, planned/legacy behavior, row counts, or SQLite and SQL Server compatibility.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
MemoryInterface._build_scenario_history_unit_statement() resolves persisted attack attempts to the logical work units used by history counters. It handles planned runs, legacy runs, seed attribution, objective-hash fallback, and deterministic selection between possible matches. This is a substantial query-building responsibility hidden inside the general memory interface.
This is M2 of three scenario-history extraction steps. Not ready yet: depends on #2766. Once that first extraction establishes the internal query boundary, move this matching logic into the same cohesive module. Wait for #2766 to land and confirm that boundary before adding help wanted.
Describe the solution you'd like
Extract only attempt-to-work-unit resolution and its directly related query construction. Keep aggregate arithmetic and aggregate-record conversion in their existing location until the next step.
- Preserve the current query shape and result-column contract consumed by
_build_scenario_history_aggregate_statement(). - Reuse backend-specific plan-unit/seed expansion and attribution expressions through the internal boundary from #2766.
- Keep exact seed-group matching, objective-hash fallback, technique matching, and deterministic ranking in one understandable unit.
- Preserve legacy and mixed batches: runs outside the plan-resolution set retain their current identity/counting behavior.
- Wire the existing aggregate builder to the extracted matching implementation without changing the public memory API.
Acceptance criteria:
- Planned and planless runs produce the same logical unit identities as before.
- Exact seed-group matches retain priority over objective-hash fallback; ambiguous matches retain the existing deterministic ordering.
- Missing or legacy technique attribution preserves current matching behavior.
- Attempts with no matching planned unit remain excluded from planned-run counters; attempts in legacy runs remain counted as before.
- Each attempt contributes the same number of rows, without join-induced duplication.
- Mixed planned/legacy batches and empty plan-resolution sets are covered.
- SQLite and SQL Server backend hooks remain compatible, and existing memory/service history results remain unchanged.
Describe alternatives you've considered, if relevant
Combining matching and aggregate-counter extraction in one change makes the most intricate SQL harder to review. Moving this logic into Python would change query cost and memory use. Prefer a focused SQL-query extraction into the module from #2766, not a separate service or a new attribution policy.
Additional context
Starting points in pyrit/memory/memory_interface.py: _build_scenario_history_unit_statement, _get_scenario_plan_unit_subqueries, _get_scenario_attempt_unit_expressions, and its caller _build_scenario_history_aggregate_statement. Relevant coverage: tests/unit/memory/memory_interface/test_interface_scenario_history.py and tests/unit/backend/test_scenario_run_service.py.
Follow doc/code/framework.md and the applicable database, Python, and test instructions. No database migration, new public API, or changed retry/success policy is intended. The next step will extract aggregate counting and typed aggregate-record construction using this matching boundary.
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 896
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 208
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/PyRIT
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
Similar issues
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
vllm-project/vllm-metal#822 ·
Maintainers usually reply within 1 day
-
vector-store
Difficulty 1/5 1-3 hours Newbie friendliness 90/100
mem0ai/mem0#7461 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug]: chunk_span_bounds and _validated_chunk_spans reject Pydantic models ChunkSpan and AudioFileOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
BasedHardware/omi#19047 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day