validator: per-validator performance tracking
Los mantenedores suelen responder en 6 días
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 25/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Necesita aclaración
- Estado de actividad
- Activo
- Stack tecnológico
- postgresql, prometheus, rust
- Área
- api, backend, observability
Línea de trabajo
Start with crates/validator/src/metrics.rs and the referenced validator RPC handlers in crates/net/rpc/src/beacon/validator.rs; inspect how duties, epoch transitions, block imports, and metrics are currently handled. The issue leaves scope, enrollment, label cardinality, and logging design undecided, so those decisions need resolution before implementation. Done would include the agreed monitoring and rewards endpoint behavior, with the listed node and client tests covering known participation, enrollment, and proposal canonicality.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Nothing in this client, or in an ethlambda beacon node, reports how each validator is doing on chain: whether its attestations were included with the right head, target and source and how late, whether its proposals became canonical, or whether its sync committee messages made it into blocks. Every other client has this somewhere. Most of them put it in the beacon node.
Code references are at bf66e86c.
What happens today
- Validator client metrics (
crates/validator/src/metrics.rs) are process-wide counters and gauges with no labels. They count what the client sent and what failed, per duty kind:ethlambda_validator_attestations_published_total(:38),ethlambda_validator_blocks_proposed_total(:94),ethlambda_validator_sync_committee_messages_published_total(:384), and the matching failure, refusal and deadline series. They do not say what the chain did with any of it, or for which validator. - Logs name the slot and validator index when a duty fails, for example
Failed to propose this slot's block(crates/validator/src/lib.rs:938). Nothing summarizes outcomes per validator per epoch. - Nothing checks inclusion. A validator that publishes on time but is never included (a late or unpropagated attestation, say) looks healthy in every series. So does a proposer whose blocks are orphaned.
- The ethlambda beacon node has no validator monitor and does not serve the standard rewards endpoints (
/eth/v1/beacon/rewards/attestations/{epoch},/rewards/blocks/{block_id},/rewards/sync_committee/{block_id}). It servesPOST /eth/v1/validator/liveness/{epoch}(crates/net/rpc/src/beacon/validator.rs:204-221), which also counts an attestation the node merely received or accepted through the API, so it does not measure inclusion.
Why it matters
A missed head or target vote is lost income, and a run of them is usually the first sign of a slow beacon node, a bad peer set or a skewed clock. Today an operator finds out from a block explorer, a day later, if they look.
What other clients do
| Client | Where and what | Read at |
|---|---|---|
| Lighthouse | Beacon node, by design ("Monitoring is in the Beacon Node", book/src/validator_monitoring.md:10-21). --validator-monitor-auto enrolls every validator that calls beacon_committee_subscriptions (beacon_node/beacon_chain/src/validator_monitor.rs:1102-1124). Per-validator head, target and source hits and misses, inclusion distance, and block and sync committee inclusion (validator_monitor.rs:790 onward; metric names in beacon_node/beacon_chain/src/metrics.rs:1036-1138). Per-validator series and logs stop at 64 validators unless the threshold is raised (book/src/validator_monitoring.md:60). The validator client exports signing counts by status and task timings, plus one per-validator gauge, vc_attestation_duty_slot (validator_client/validator_metrics/src/lib.rs:79, :276-282) |
unstable @ 11813cfb |
| Teku | --validators-performance-tracking-mode (LOGGING, METRICS, ALL, NONE; default ALL) (teku/src/main/java/tech/pegasys/teku/cli/options/ValidatorOptions.java:75-82, validator/api/src/main/java/tech/pegasys/teku/validator/api/ValidatorPerformanceTrackingMode.java:34). The tracker is beacon/validator/src/main/java/tech/pegasys/teku/validator/coordinator/performance/DefaultPerformanceTracker.java, in the beacon/validator module. It reports attestation inclusion distance and correct target and head counts (:302-373), block production and sync committee participation |
master @ 62a54b96 |
| Prysm | Validator client. At each epoch's end it logs a Previous epoch voting summary per validator and sets per-key gauges such as validator_correctly_voted_source, _target, _head and validator_balance (validator/client/metrics.go:86-146, :248-301, :376). The data comes from Prysm's own POST /prysm/validators/performance, and the client skips it when the node's version string does not say Prysm (validator/client/beacon-api/beacon_api_beacon_chain_client.go:307-332) |
develop @ 0c1e3d24 |
| Nimbus | Beacon node. Logs Block proposal included and missing, Previous epoch attestation included and missing, and Attestation failed to match target and head per monitored validator (beacon_chain/validators/validator_monitor.nim:525-564), with automatic enrollment (addAutoMonitor, :298) |
unstable @ 9bed158a |
| Lodestar | Beacon node, packages/beacon-node/src/chain/validatorMonitor.ts. Local validators are registered through registerLocalValidator (:346). Per-epoch logs such as Previous epoch attestation (:782) are at debug unless validatorMonitorLogs is set (:130, :305) |
unstable @ 77088eb7 |
Where it belongs
Mostly in the beacon node, with a small part in the client:
- The node has the data. At each epoch transition its state holds every validator's timely source, target and head flags for the previous epoch. Each imported block says who proposed it and which attestations and sync committee bits it carries, which gives inclusion distance and sync participation. A client can only get this through endpoints the ethlambda node does not serve, or through a private one, as Prysm does.
- A node-side monitor works for any client pointed at the node, and can watch validators that are not attached to it.
- The client knows what it attempted, which the node does not: which duties it had, which it signed, and which failed before reaching any node.
Proposal
- Beacon node: a validator monitor.
- Enrollment: automatic from
beacon_committee_subscriptionsandprepare_beacon_proposer(crates/net/rpc/src/beacon/validator.rs:72,:76), plus an explicit pubkey list. - At each epoch transition: per monitored validator, previous-epoch source, target and head hits from the participation flags, and the effective balance and
slashedflag. - At each block import: inclusion distance of monitored validators' attestations and their sync committee bits.
- Per monitored proposer duty: whether the slot ended up with a canonical block, and whether the node saw a block for it at all, which separates a missed proposal from an orphaned one.
- Output: Prometheus series with a
validatorlabel up to a cap (Lighthouse's 64) and aggregate series beyond it, plus one log line per validator per epoch at info.
- Enrollment: automatic from
- Beacon node, separately: the standard rewards endpoints, so any client or tool can compute the same from ethlambda's node.
- Validator client: per-validator outcomes of what it attempted.
- Proposals published or failed, attestations published or failed, and sync committee messages, with a capped
validatorlabel. - A proposal check a few slots later through
GET /eth/v1/beacon/headers/{slot}, which ethlambda's node serves, to see whether the published block is the canonical one at its slot. - Against a node that serves the rewards endpoints, attestation rewards for the client's own indices.
- Proposals published or failed, attestations published or failed, and sync committee messages, with a capped
- #765 (per-block ClickHouse telemetry, open, opt-in) could later carry per-validator rows, but Prometheus and logs should come first.
To decide
- Scope and title. Most of the work lands in beacon node crates. Keep one issue, or split the node's monitor and its rewards endpoints into issues of their own and keep the client's part here?
- Label cardinality. A
validatorlabel on thousands of keys (#797) is a lot of series. Cap as Lighthouse does, or keep per-validator detail in logs only? - Enrollment. Automatic only, explicit only, or both.
- Inclusion distance. Since altair the participation flags record timeliness only, so inclusion distance has to come from the attestations in each block.
- Logs. One line per validator per epoch (Prysm, Nimbus) or an aggregate line with per-validator detail at debug (Lodestar).
Tests
- Node: a fixture state with known participation flags gives the expected hits and misses for enrolled validators. A proposer duty whose slot has no canonical block counts as a miss.
- Node: auto-enrollment after a
beacon_committee_subscriptionsrequest. - Client:
MockBeaconNodeanswering/eth/v1/beacon/headers/{slot}with a 404, or with a root other than the published block's, marks the client's proposal as not canonical.
- Lenguaje dominante
- Rust
- Estrellas
- 86
- Forks
- 34
- Merge medio
- 9 d 20 h
- PR fusionados (30 d)
- 77
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de lambdaclass/ethlambda
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
lambdaclass/ethlambda#850 ·
Los mantenedores suelen responder en 6 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
lambdaclass/ethlambda#777 · 1 reacción ·
Los mantenedores suelen responder en 6 días
-
Run all gossip validation testsQuizá libre de nuevo @dicethedev la tomó hace 9 días y no hay ningún pull request abierto. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
lambdaclass/ethlambda#725 · 2 comentarios ·
Los mantenedores suelen responder en 6 días
-
Handle malformed DB dataQuizá libre de nuevo @dicethedev la tomó hace 115 días y no hay ningún pull request abierto. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
lambdaclass/ethlambda#529 · 4 comentarios ·
Los mantenedores suelen responder en 6 días
-
Document architectural choicesQuizá libre de nuevo @dicethedev la tomó hace 115 días y no hay ningún pull request abierto. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
lambdaclass/ethlambda#521 · 2 comentarios ·
Los mantenedores suelen responder en 6 días
Todos los issues de lambdaclass/ethlambda
Issues similares
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
bytecodealliance/wrpc#1785 ·
Los mantenedores suelen responder en 1 día
-
2 advisory ignore entries no longer match anything (`quick-xml` already patched in `Cargo.lock`)Abierto
Dificultad 1/5 Menos de una hora Aptitud para principiantes 85/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
longbridge/nsh#2758 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
antithesishq/bombadil#473 ·
Los mantenedores suelen responder en 2 días
-
bench/api.sh exec test fails with curl exit 52, and --ignore-failure still records the timingsAbiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
libracoder/kiri#669 ·
Los mantenedores suelen responder en 1 día