Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

P3 scoring: BlockFLow validation (P3-SCR), on-chain median, scenario-driven scoring redesign

Open
  • Is something wrong?
#39 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
machine-learning, python, solidity
Domain
blockchain, databases, machine-learning, testing

Research direction

Start with MECHANISM_DESIGN §6 and Developer/issues/scoring-mechanism.md, then locate the shipped BlockFLow scoring implementation and its pytest integration tests. Validate poison detection, duplicate fold-in, permutation averaging, and sequential fold-in across the stated cases; inspect the finalizeEvaluation path for the on-chain auditor-batch median and eligibility gate. Done means the tests cover the specified scoring properties and the median score is the canonical basis for eligibility, rewards, and S3 slashing.

Written by the indexing model from the issue text.

Description

enhancement

Part of the P3 mechanism-design push (#36).

Summary

Two workstreams before scoring can safely drive rewards and slashing:

  1. Validate the shipped BlockFLow-inspired scoring (WP 2.2 ✅) empirically — P3-SCR. High correctness risk in the subtle properties (sequential fold-in, permutation averaging).
  2. Scenario-driven scoring redesign — separate eligibility from utility instead of one test shard / one accuracy number / one average threshold (issues/scoring-mechanism.md).

Scope (from MECHANISM_DESIGN §6)

  • P3-SCR validation: poison detection (label-flip, sign-flip, scaling, garbage), duplicate discounting via fold-in, cosine-to-consensus / norm-bound / marginal-gain gates at multiple ε values; pytest integration tests for permutation averaging and sequential fold-in
  • Compute the median across the auditor batch on-chain at finalizeEvaluation so the canonical per-model score exists on-chain as both the S3 slashing reference and the reward basis (today eligibility voting exists but no canonical score is derived)
  • Eligibility: median ≥ passScore gates T1 aggregation; below ⇒ rejected, zero reward
  • S3 deviation threshold: start wide, warning-only shadow mode for the first weeks of DevNet 2.0, then tighten (avoid punishing honest variance from heterogeneous audit test sets)

Threat coverage

In scope: crude poisoning, Sybil clients (spam costs compute, earns nothing below pass score), score inflation (median-bounded). Out of scope for 2.0: backdoors (RES-2), >50% auditor collusion.

Design sources

MECHANISM_DESIGN §6 · issues/scoring-mechanism.md · whitepaper-summary §4.4, §8 gap items 3, 11, 14 · rejected-ideas/tknn-shapley.md

Roadmap anchors: P3-SCR (validation), WP 2.2 (implementation, shipped)

Dominant language
Python
Stars
4
Forks
7
Avg merge
3d 5h
Merged PRs (30d)
34

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from InfiniteZeroFoundation/DevNet

All issues in InfiniteZeroFoundation/DevNet

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.