[Epic][TS PBT] Build and evaluate a property-aware PBT–USVM feedback system
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 25/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- kotlin, typescript
- 領域
- compilers, devtools, testing-qa
調査の方向性
Start with the contract gate in #384, then read the existing foundations (#346-#349) and the symbolic projection, search, replay, and baseline work (#351-#354). Confirm how the native JacoDB TypeScript frontend and original runtime are used, then define the integration around the stated backend interface. Done means comparable PBT_ONLY, SYMBOLIC_ONLY, and HYBRID runs with replayed counterexamples, recorded results, and documented capabilities and limits.
索引モデルが issue の本文から書いたものです。
説明
Intended result
Deliver the strongest evidence-backed version of the TypeScript PBT/USVM work: reuse human-written properties and generator semantics, infer property-relevant behavioral relations from concrete executions, challenge them symbolically, and return validated inputs to further generation and shrinking.
Coverage feedback is mandatory infrastructure. Observed relations are a central research mechanism. Generator-choice search, metamorphic relations, bounded command models and context-scoped helper summaries are included in the full delivery through bounded, working implementations and separate evaluation. They are not deferred as unspecified future work.
The full epic completes with an implemented, evaluated and reproducible result; scientific superiority is a hypothesis, not an acceptance criterion that permits hiding negative results.
Delivery stages
The full scope above is retained, but it is delivered through independently reviewable checkpoints. The first checkpoint does not wait for all extensions or the final benchmark corpus.
| Stage | Owner | Completion boundary |
|---|---|---|
| Core implementation and development pilot | #405, with #355 owning integration | A bounded end-to-end loop, exact coverage feedback, original-oracle replay, returned-input generation and equal-budget development comparisons; a recorded assessment of feasibility and empirical promise |
| Bounded extensions | #400, #401, #402, #403 | Working generator-choice, multi-execution, command-model and helper-context extensions on the same core contracts |
| Final evaluation and paper | #356, #357, #404 | Frozen held-out evaluation of the core and each extension, limitations, manuscript and reproducible artifact |
The core starts with supported bounded numeric/scalar inputs, tuples and dense bounded arrays; selected arguments, returns and explicit intermediate observation points; a small frozen relation vocabulary; and deterministic target scheduling. This is a declared capability boundary, not permission to approximate unsupported JavaScript behavior silently. Ordinary multi-call predicates remain usable wherever already supported; dedicated cross-execution inference belongs to #401.
#405 separates functional completion from evidence of benefit. A reproducible negative or inconclusive pilot can complete that checkpoint with a documented diagnosis and next decision; it cannot be called a demonstrated improvement. The full epic and its required extensions remain open. A later reduction of full scope must be an explicit roadmap change, never an inference from a negative run.
Dependencies are completion gates. Literature, corpus selection, protocol design and extension interface design may start early. Core component issues close on their stated bounded contracts; extension-specific implementation and acceptance are owned by #400–#403, not retroactively added to those core issues.
Semantic contract
- Kotlin owns common artifacts and orchestration; fast-check is the first concrete backend. Reuse completed #346–#350 and #384, the native TypeScript frontend, original TypeScript oracles and existing tagged values.
- Distinguish declared input support/preconditions, generator probability bias, concrete observations, empirical hypotheses and proved facts. Search for a violation of the declared predicate; never assume that predicate or a sampled relation as an unconditional fact.
- Challenge a hypothesis using declared domain AND admitted precondition AND reach(point/context) AND NOT(hypothesis). Preserve context/guards and value identity. Refuting a hypothesis is not automatically a defect.
- Every finding must replay in the original runtime. Unsupported execution, solver uncertainty, invalid models, timeouts and failed extraction remain visible. No finite unsuccessful search proves a property.
- Covered edges remain searchable. Input-dependent faults, relational behavior and different states on one path remain relevant after coverage plateaus.
- Speculative restrictions/substitutions are labeled experimental assumptions, with replay and a positive bounded unrestricted allocation under the same total deadline. Fallback alone is not a completeness guarantee.
- Preserve #384 sample isolation; bounded stateful sequences explicitly preserve state only within one invocation.
Work packages
| Package | Owning issues | Required output |
|---|---|---|
| Delivered foundation | #346, #347, #348, #349, #350, #384 | Existing models, concrete execution, coverage, mapping and shared semantics |
| Direct symbolic baseline | #351, #352 | Existing scoped PR #387; declared-domain and original-predicate search |
| User-authored semantics | #395 | Original predicates/assertions and support-preserving generator relationships |
| Replay, shrinking and campaign shell | #353, #354 | Original-runtime validation, target-preserving reduction, common budgets/artifacts and controls |
| Exact branch signal | #382 | Real-runtime branch-to-CFG mapping; semantic coverage usable by search |
| Feedback integration | #355 | Bounded core PBT → observations → symbolic challenge → replay → PBT cycle |
| First research checkpoint | #405 | Core functional evidence, development comparisons and an explicit proceed/refine/reassess decision |
| Observations | #396 | Bounded original-runtime values with property/run/point provenance |
| Relational inference | #397 | Property-focused, guarded hypotheses with evidence and contradictions |
| Symbolic challenge binding | #398 | Exact value/location/time/context projection and useful-prefix targets |
| Search scheduling | #399 | Auditable assertion/coverage/hypothesis scheduling under one budget |
| Generator-choice search | #400 | A working bounded dependent-generator subset and original-generator replay |
| Metamorphic execution | #401 | Correlated multiple executions and original relational oracles |
| Stateful command models | #402 | Bounded command sequences, guards, state invariants and reduction |
| Helper summaries | #403 | Stage-two context-specific relations, challenge and refinement; independent of core completion |
| Corpus and evaluation | #356, #357 | Real properties, validated faults, development/held-out split and causal ablations |
| Novelty and paper | #404 | Primary-source comparison, claim/evidence ledger, manuscript and reproducible package |
Execution order and orchestration
- Start corpus selection (#356) and literature/claim design (#404) immediately. Preserve the already reviewed foundation and the narrow #387 delivery; new research mechanisms land in their owning follow-ups.
- Complete #351/#352 and implement #353/#354. In parallel, establish #395, #396 and #382.
- Deliver the bounded #397/#398 contracts, then #399 and #355's core repeated loop. Use the development manifest from #356 and protocol/nearest-work inputs from #357/#404 without waiting for those issues to close.
- Complete #405: check functional causality, run the equal-budget development comparisons, account for all overhead, and record the next decision. Do not make #400–#403 prerequisites for this checkpoint or expand the pilot to rescue an unfavorable result.
- Use the pilot's diagnosis to implement and integrate #400, #401, #402 and #403 on the same budget/oracle contracts. Each remains required for the full epic, with a nonempty supported subset, conformance evidence and ablation. Their completion depends on the pilot assessment; early interface design is allowed.
- Freeze final templates, bounds, scheduler, corpus exclusions and settings on development data, then execute #357 and complete #404. Final results do not tune the method or select favorable benchmarks.
Use one PR per independently reviewable contract/mechanism where practical, with linked issue, focused behavior tests, actual checked revision and reproduction commands. Keep the dependency graph acyclic; distinguish readiness to start work from gates on final integration. Do not mark an issue implemented solely because its plan or fixture exists.
Evaluation and completion gates
- Same original user oracles are used in PBT-only, symbolic-only, sequential, coverage-only, relations-only and combined modes. Add observation-independent template, unfocused-inference, no-return-loop, output-novelty and extension ablations in #357.
- Total budgets include observation/inference/mapping, frontend/adapter work, search, replay and reduction; report setup consistently. Measure distinct confirmed faults and time to confirmation, not coverage alone.
- Include held-out real properties, known defects and separately reported validated mutants; saturated-coverage, misleading-hypothesis, generator-dependence, metamorphic and stateful cases.
- Each required bounded extension works end to end. Unsupported-only placeholders or toy wins cannot stand in for the full scope or its evaluation.
- Raw artifacts expose observations, inferred expressions, evidence provenance, mappings, target decisions, replay outcomes, returned seeds, costs and limitations. Regeneration commands reproduce the paper's tables.
- Nearest-work analysis covers coverage-guided PBT, dynamic invariants, static/dynamic testing, hypothesis falsification, generator choices and model-based testing. The supplied Go/gopter thesis includes coverage, input corpora and behaviorKey novelty; reuse of values alone is not a novelty claim.
- Preserve negative results and distinguish functional completion from measured improvement. Keep unknown-call policy work (#360/#385), the ICCQ manuscript and IFDS/type-inference research separate.
Arbitrary JavaScript closure serialization, universal invariant discovery, arbitrary heap/environment modeling, concurrency/async schedule exploration and a general plugin platform are outside the required bounded implementation. This does not exclude the explicitly listed structured generators, metamorphic relations or within-invocation stateful command models.
- 主要言語
- Kotlin
- スター
- 33
- フォーク
- 27
- 平均マージ
- 2日 20時間
- マージ済み PR(30日)
- 10
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
UnitTestBot/usvm のほかの issue
-
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
UnitTestBot/usvm#467 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 38/100
UnitTestBot/usvm#465 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 55/100
UnitTestBot/usvm#462 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
UnitTestBot/usvm#457 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
UnitTestBot/usvm#440 ·
メンテナーはふだん 1 日以内に返信
UnitTestBot/usvm の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 64/100
maplibre/maplibre-native-ffi#792 ·
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 74/100
メンテナーはふだん 1 日以内に返信
-
an:enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 74/100
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100