[TS PBT][Checkpoint] Validate the bounded core feedback loop and assess its research value
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 25/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- javascript, kotlin, typescript
- 領域
- testing-qa
調査の方向性
The issue names no files, tests, or entry points. Start by reading the core integration and dependency work in #355, along with the artifacts from #356, #357, and #404; completion requires the bounded functional gates, frozen development comparisons, reproducible evidence, and a justified next-action decision.
索引モデルが issue の本文から書いたものです。
説明
Part of #345. This is the first independently completable research checkpoint. #355 owns core integration; its dependency chain includes #353/#354, #382 and #395–#399. #356 supplies a development manifest, #357 supplies the pilot protocol and #404 supplies nearest-work/claim analysis as early artifacts; none of those three issues needs to close first. #400–#403 are subsequent extensions, not prerequisites.
Objective
Establish whether a bounded, property-aware PBT → observations → symbolic challenge → original-runtime replay → PBT cycle works, and measure whether its empirical promise justifies the next research steps. Separate implementation feasibility from comparative benefit. Full #345 delivery remains broader than this checkpoint.
Fixed core scope
- Reuse original human-written predicates/assertions, declared preconditions and generator support. Start with supported bounded numeric/scalar inputs, tuples and dense bounded arrays; include a directly representable dependent input relationship without requiring generator-choice search.
- Observe selected arguments/returns and explicit intermediate points with stable property, source, invocation and pre/post identity. Use a small documented vocabulary of equality/order, bounded affine, length, pre/post and simple guarded relations tied to the user's assertion.
- Preserve actual JavaScript semantics and classify unsupported bindings/values explicitly. No arbitrary heap/generator execution, command-model framework, dedicated cross-execution inference or helper-summary refinement is required.
- Use deterministic scheduling of original-property, exact mapped branch-coverage and hypothesis targets under one total deadline. Existing ordinary multi-call predicates remain usable wherever supported.
- Replay every candidate in original TypeScript, shrink eligible witnesses while preserving the target, and return useful inputs to a bounded generator-consistent neighborhood policy for a subsequent PBT round.
Functional acceptance
- Real PBT runs produce a nontrivial property-relevant relation not implied by input support/preconditions and not merely a restatement of the original property; that relation changes an actual USVM target.
- A replay-confirmed hypothesis-refutation witness changes subsequent generated inputs and relation evidence. Re-executing an unchanged example list alone is insufficient.
- A curated fault fixture produces an original-property failure via an observation-derived challenge after coverage saturates. Keep it separate from a useful passing hypothesis refutation; do not use a false specification as an implementation defect or hard-code the fault target into inference.
- Independently changed real-runtime branch evidence changes a coverage target through #382/#399. Statement-only mapping does not satisfy this gate.
- Source/context/time mismatches, misleading hypotheses, unsupported cases, timeouts and budget exhaustion have explicit outcomes. Preserve original-property search and concrete validation budget.
- At least one eligible external witness is reduced and still reproduces its original target. Unsupported shrinking elsewhere remains visible.
- One reproducible command and pinned artifacts expose input lineage, observations, inferred formulas, exact bindings, target choices, replay, returned inputs and all phase costs.
The curated fault demonstrates functionality and causal use of an observation. It does not establish an advantage over direct symbolic search of the same property.
Development comparison
- Freeze a versioned manifest and comparison settings before measuring outcomes. Include at least two independent property families, an imported real fast-check suite, saturated-coverage cases, a misleading/no-benefit case and correct implementations. Report project concentration and adaptation cost. All these cases remain development data.
- Run PBT_ONLY, SYMBOLIC_ONLY for the same original property, SEQUENTIAL, COVERAGE_FEEDBACK, RELATION_FEEDBACK and COMBINED_FEEDBACK with identical declared support and total budgets on explicit common supported subsets.
- Include planned contrasts for observation-independent templates with the same vocabulary/target budget, property-focused versus unfocused inference, no-return-loop, and output-novelty selection. Run native fast-check on eligible suites to expose adaptation overhead. Do not require a Cartesian product of controls.
- Count original-oracle-confirmed implementation faults and time to confirmation; report hypothesis-only refutations and false specifications separately. Separate real defects, validated mutants and curated functional cases.
- Include observation/inference/mapping, startup, search, replay, feedback and shrinking costs. Use predeclared independent seeds/repetitions, report uncertainty and timeouts, preserve regressions/no-gain cases and retain the supported denominator. Never claim benefit from coverage or one selected seed alone.
- Keep final held-out projects/families out of pilot tuning. Version later pilot revisions and retain prior results instead of selecting a favorable comparison retrospectively.
Assessment and completion
Record pinned code/configuration/manifest revisions, reproduction commands, functional evidence, comparison tables, nearest-work implications and the main limiting costs/capabilities. Choose and justify a next action: proceed with the planned extensions; refine a specific core mechanism in a separately recorded development iteration; or reassess the research claim/scope in an explicit roadmap decision.
This checkpoint closes when the bounded functional gates, planned development comparisons and assessment are complete. A negative or inconclusive comparison is a valid completed result; no positive effect size, statistically significant win or general superiority claim is required. Unimplemented core behavior or missing required measurements is not a completed pilot and stays open with explicit blockers.
Closure does not close #345, #356, #357 or #404, waive #400–#403, or authorize submission/publication. The full roadmap retains those required deliverables. Final empirical claims come from the frozen held-out evaluation, not this development pilot.
- 主要言語
- Kotlin
- スター
- 33
- フォーク
- 27
- 平均マージ
- 2日 20時間
- マージ済み PR(30日)
- 10
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
UnitTestBot/usvm のほかの issue
-
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
UnitTestBot/usvm#467 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 38/100
UnitTestBot/usvm#465 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 55/100
UnitTestBot/usvm#462 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
UnitTestBot/usvm#457 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
UnitTestBot/usvm#440 ·
メンテナーはふだん 1 日以内に返信
UnitTestBot/usvm の issue をすべて見る
似ている issue
-
good first issue help wanted
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
-
Bookmarking an article that is already bookmarked under its redirect URL deletes both bookmarks対応中かも @aakarshitv が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
kiwix/kiwix-android#5176 ·
メンテナーはふだん 1 日以内に返信
-
Source: c7 Type: bug
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
-
難易度 1/5 1時間未満 初心者へのやさしさ 90/100
navikt/esyfo-narmesteleder#654 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
メンテナーはふだん 1 日以内に返信