Hacktoberfest 2026:メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

[Epic][TS PBT] Build and evaluate a property-aware PBT–USVM feedback system

オープン
#345 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
kotlin, typescript

調査の方向性

Start with the contract gate in #384, then read the existing foundations (#346-#349) and the symbolic projection, search, replay, and baseline work (#351-#354). Confirm how the native JacoDB TypeScript frontend and original runtime are used, then define the integration around the stated backend interface. Done means comparable PBT_ONLY, SYMBOLIC_ONLY, and HYBRID runs with replayed counterexamples, recorded results, and documented capabilities and limits.

索引モデルが issue の本文から書いたものです。

説明

Intended result

Deliver the strongest evidence-backed version of the TypeScript PBT/USVM work: reuse human-written properties and generator semantics, infer property-relevant behavioral relations from concrete executions, challenge them symbolically, and return validated inputs to further generation and shrinking.

Coverage feedback is mandatory infrastructure. Observed relations are a central research mechanism. Generator-choice search, metamorphic relations, bounded command models and context-scoped helper summaries are included in the full delivery through bounded, working implementations and separate evaluation. They are not deferred as unspecified future work.

The full epic completes with an implemented, evaluated and reproducible result; scientific superiority is a hypothesis, not an acceptance criterion that permits hiding negative results.

Delivery stages

The full scope above is retained, but it is delivered through independently reviewable checkpoints. The first checkpoint does not wait for all extensions or the final benchmark corpus.

Stage Owner Completion boundary
Core implementation and development pilot #405, with #355 owning integration A bounded end-to-end loop, exact coverage feedback, original-oracle replay, returned-input generation and equal-budget development comparisons; a recorded assessment of feasibility and empirical promise
Bounded extensions #400, #401, #402, #403 Working generator-choice, multi-execution, command-model and helper-context extensions on the same core contracts
Final evaluation and paper #356, #357, #404 Frozen held-out evaluation of the core and each extension, limitations, manuscript and reproducible artifact

The core starts with supported bounded numeric/scalar inputs, tuples and dense bounded arrays; selected arguments, returns and explicit intermediate observation points; a small frozen relation vocabulary; and deterministic target scheduling. This is a declared capability boundary, not permission to approximate unsupported JavaScript behavior silently. Ordinary multi-call predicates remain usable wherever already supported; dedicated cross-execution inference belongs to #401.

#405 separates functional completion from evidence of benefit. A reproducible negative or inconclusive pilot can complete that checkpoint with a documented diagnosis and next decision; it cannot be called a demonstrated improvement. The full epic and its required extensions remain open. A later reduction of full scope must be an explicit roadmap change, never an inference from a negative run.

Dependencies are completion gates. Literature, corpus selection, protocol design and extension interface design may start early. Core component issues close on their stated bounded contracts; extension-specific implementation and acceptance are owned by #400–#403, not retroactively added to those core issues.

Semantic contract

  • Kotlin owns common artifacts and orchestration; fast-check is the first concrete backend. Reuse completed #346–#350 and #384, the native TypeScript frontend, original TypeScript oracles and existing tagged values.
  • Distinguish declared input support/preconditions, generator probability bias, concrete observations, empirical hypotheses and proved facts. Search for a violation of the declared predicate; never assume that predicate or a sampled relation as an unconditional fact.
  • Challenge a hypothesis using declared domain AND admitted precondition AND reach(point/context) AND NOT(hypothesis). Preserve context/guards and value identity. Refuting a hypothesis is not automatically a defect.
  • Every finding must replay in the original runtime. Unsupported execution, solver uncertainty, invalid models, timeouts and failed extraction remain visible. No finite unsuccessful search proves a property.
  • Covered edges remain searchable. Input-dependent faults, relational behavior and different states on one path remain relevant after coverage plateaus.
  • Speculative restrictions/substitutions are labeled experimental assumptions, with replay and a positive bounded unrestricted allocation under the same total deadline. Fallback alone is not a completeness guarantee.
  • Preserve #384 sample isolation; bounded stateful sequences explicitly preserve state only within one invocation.

Work packages

Package Owning issues Required output
Delivered foundation #346, #347, #348, #349, #350, #384 Existing models, concrete execution, coverage, mapping and shared semantics
Direct symbolic baseline #351, #352 Existing scoped PR #387; declared-domain and original-predicate search
User-authored semantics #395 Original predicates/assertions and support-preserving generator relationships
Replay, shrinking and campaign shell #353, #354 Original-runtime validation, target-preserving reduction, common budgets/artifacts and controls
Exact branch signal #382 Real-runtime branch-to-CFG mapping; semantic coverage usable by search
Feedback integration #355 Bounded core PBT → observations → symbolic challenge → replay → PBT cycle
First research checkpoint #405 Core functional evidence, development comparisons and an explicit proceed/refine/reassess decision
Observations #396 Bounded original-runtime values with property/run/point provenance
Relational inference #397 Property-focused, guarded hypotheses with evidence and contradictions
Symbolic challenge binding #398 Exact value/location/time/context projection and useful-prefix targets
Search scheduling #399 Auditable assertion/coverage/hypothesis scheduling under one budget
Generator-choice search #400 A working bounded dependent-generator subset and original-generator replay
Metamorphic execution #401 Correlated multiple executions and original relational oracles
Stateful command models #402 Bounded command sequences, guards, state invariants and reduction
Helper summaries #403 Stage-two context-specific relations, challenge and refinement; independent of core completion
Corpus and evaluation #356, #357 Real properties, validated faults, development/held-out split and causal ablations
Novelty and paper #404 Primary-source comparison, claim/evidence ledger, manuscript and reproducible package

Execution order and orchestration

  1. Start corpus selection (#356) and literature/claim design (#404) immediately. Preserve the already reviewed foundation and the narrow #387 delivery; new research mechanisms land in their owning follow-ups.
  2. Complete #351/#352 and implement #353/#354. In parallel, establish #395, #396 and #382.
  3. Deliver the bounded #397/#398 contracts, then #399 and #355's core repeated loop. Use the development manifest from #356 and protocol/nearest-work inputs from #357/#404 without waiting for those issues to close.
  4. Complete #405: check functional causality, run the equal-budget development comparisons, account for all overhead, and record the next decision. Do not make #400–#403 prerequisites for this checkpoint or expand the pilot to rescue an unfavorable result.
  5. Use the pilot's diagnosis to implement and integrate #400, #401, #402 and #403 on the same budget/oracle contracts. Each remains required for the full epic, with a nonempty supported subset, conformance evidence and ablation. Their completion depends on the pilot assessment; early interface design is allowed.
  6. Freeze final templates, bounds, scheduler, corpus exclusions and settings on development data, then execute #357 and complete #404. Final results do not tune the method or select favorable benchmarks.

Use one PR per independently reviewable contract/mechanism where practical, with linked issue, focused behavior tests, actual checked revision and reproduction commands. Keep the dependency graph acyclic; distinguish readiness to start work from gates on final integration. Do not mark an issue implemented solely because its plan or fixture exists.

Evaluation and completion gates

  • Same original user oracles are used in PBT-only, symbolic-only, sequential, coverage-only, relations-only and combined modes. Add observation-independent template, unfocused-inference, no-return-loop, output-novelty and extension ablations in #357.
  • Total budgets include observation/inference/mapping, frontend/adapter work, search, replay and reduction; report setup consistently. Measure distinct confirmed faults and time to confirmation, not coverage alone.
  • Include held-out real properties, known defects and separately reported validated mutants; saturated-coverage, misleading-hypothesis, generator-dependence, metamorphic and stateful cases.
  • Each required bounded extension works end to end. Unsupported-only placeholders or toy wins cannot stand in for the full scope or its evaluation.
  • Raw artifacts expose observations, inferred expressions, evidence provenance, mappings, target decisions, replay outcomes, returned seeds, costs and limitations. Regeneration commands reproduce the paper's tables.
  • Nearest-work analysis covers coverage-guided PBT, dynamic invariants, static/dynamic testing, hypothesis falsification, generator choices and model-based testing. The supplied Go/gopter thesis includes coverage, input corpora and behaviorKey novelty; reuse of values alone is not a novelty claim.
  • Preserve negative results and distinguish functional completion from measured improvement. Keep unknown-call policy work (#360/#385), the ICCQ manuscript and IFDS/type-inference research separate.

Arbitrary JavaScript closure serialization, universal invariant discovery, arbitrary heap/environment modeling, concurrency/async schedule exploration and a general plugin platform are outside the required bounded implementation. This does not exclude the explicitly listed structured generators, metamorphic relations or within-invocation stateful command models.

主要言語
Kotlin
スター
33
フォーク
27
平均マージ
2日 20時間
マージ済み PR(30日)
10

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

UnitTestBot/usvm のほかの issue

UnitTestBot/usvm の issue をすべて見る

似ている issue

Kotlin の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。