Subscription gap metric reads the end of the same stream once for each subscription
Los mantenedores suelen responder en 7 días
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bien especificado
- Estado de actividad
- Estancado
- Stack tecnológico
- csharp
- Área
- observability
Línea de trabajo
Start with src/Core/src/Eventuous.Subscriptions/Diagnostics/SubscriptionMetrics.cs, focusing on ObserveGapValues and GetGap, then trace how measures are registered in SubscriptionRegistrationExtensions.cs. Review the SQL and KurrentDB measure implementations listed in the issue to see how they identify an end-of-stream source. Done means each shared source is read once per collection while subscription-id tags and custom implementations without a key retain their stated behavior; linked pull request 615 is already open.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Is your feature request related to a problem? Please describe.
AddEventuousSubscriptions() registers the subscription gap gauges in SubscriptionMetrics. On each metric collection, ObserveGapValues calls the GetSubscriptionEndOfStream delegate of every measured subscription, one after another. Each delegate reads the end of its stream from the store.
When several subscriptions read the same source, every call returns the same value, but the store gets one request for each subscription. This happens in the core SubscriptionMetrics, so it affects every store that supports the gap metric:
| Store | End-of-stream request for an all-stream subscription |
|---|---|
| SQL Server | SELECT MAX(GlobalPosition) FROM <schema>.Messages |
| PostgreSQL | select max(global_position) from <schema>.messages |
| SQLite | SELECT MAX(global_position) FROM <messages table> |
| KurrentDB | ReadAllAsync(Direction.Backwards, Position.End, 1), a gRPC read of the last event in $all |
For example, an application has six all-stream subscriptions on one SQL Server store. Each collection then sends the same query six times. The position column is the clustered primary key, so each SQL query is cheap. For KurrentDB, each call is a network round trip. In all cases, the number of requests grows with the number of subscriptions and with the export frequency:
| Metric export interval | Subscriptions | End-of-stream requests |
|---|---|---|
| 60 seconds | 6 | 6 each minute |
| 200 milliseconds | 6 | about 30 each second |
A second, related problem: GetGap runs each delegate synchronously (GetAwaiter().GetResult()) on the collection thread, with a 500 ms timeout for each subscription. When the store is slow, one collection can block for up to 500 ms multiplied by the number of subscriptions. This also applies to every store.
Observed with Eventuous 0.17.0 (Eventuous.SqlServer, Eventuous.Diagnostics.OpenTelemetry). The other stores are from a reading of the dev branch source.
Describe the solution you'd like
Read each end-of-stream source one time per collection, and share the result between the subscriptions that read that source. Make the change in SubscriptionMetrics, so that it applies to all stores.
A possible shape:
IMeasuredSubscriptionexposes a key for its end-of-stream source in addition to the delegate. Subscriptions that read the same source return the same key. For example, a SQL all-stream subscription returns the store type, the connection, and the schema. A stream subscription adds the stream name. A KurrentDB$allsubscription returns the client and$all.SubscriptionMetricsgroups the measures by key. On each collection, it calls one delegate for each key and uses the result for every subscription in the group. Thesubscription-idtag stays the same, so the exported series do not change.- Optionally,
SubscriptionMetricsruns the remaining calls concurrently, with one overall timeout, rather than one blocking call after another.
The key could be optional. A subscription with no key keeps the current behavior, so custom IMeasuredSubscription implementations continue to work.
Additional context
The relevant code:
src/Core/src/Eventuous.Subscriptions/Diagnostics/SubscriptionMetrics.cs:ObserveGapValuesandGetGap.src/Core/src/Eventuous.Subscriptions/Registrations/SubscriptionRegistrationExtensions.cs: registers oneGetSubscriptionEndOfStreamfor each measured subscription.src/Relational/src/Eventuous.Sql.Base/Subscriptions/SqlSubscriptionBase.cs:GetSubscriptionEndOfStream, which SQL Server, PostgreSQL, and SQLite share.PrepareEndOfStreamCommandin the all-stream and stream subscriptions ofEventuous.SqlServer,Eventuous.Postgresql, andEventuous.Sqlite.src/KurrentDB/src/Eventuous.KurrentDB/Subscriptions/Diagnostics/:AllStreamSubscriptionMeasure,StreamSubscriptionMeasure, andBaseSubscriptionMeasure.
- Lenguaje dominante
- C#
- Estrellas
- 512
- Forks
- 99
- Merge medio
- 5 d 6 h
- PR fusionados (30 d)
- 5
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de Eventuous/eventuous
-
MongoDB integration tests fail on newer GitHub Actions runner kernelsPosiblemente ocupada @nmummau la tomó hace 3 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
Eventuous/eventuous#626 · 2 comentarios ·
Los mantenedores suelen responder en 7 días
-
`dotnet format style --verify-no-changes` fails with naming violations and unnecessary importsPosiblemente ocupada @nmummau la tomó hace 3 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
Eventuous/eventuous#624 · 1 comentario ·
Los mantenedores suelen responder en 7 días
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
Eventuous/eventuous#627 · 1 comentario ·
Los mantenedores suelen responder en 7 días
-
SQL Server: `append_events` does not need the `@inserted` table variablePosiblemente ocupada @nmummau la tomó hace 9 días. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 20/100
Eventuous/eventuous#619 · 1 comentario ·
Los mantenedores suelen responder en 7 días
-
SQL Server: stream reads do one key lookup per event, and `IDX_EventsStream` duplicates `UQ_StreamIdAndStreamPosition`Posiblemente ocupada @nmummau la tomó hace 9 días. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 18/100
Eventuous/eventuous#618 · 1 comentario ·
Los mantenedores suelen responder en 7 días
Todos los issues de Eventuous/eventuous
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
Nimblesite/SharpLsp#468 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
elastic/ecs-dotnet#575 ·
-
V4 alert-rule reads have no scope gate, so any authenticated credential can list alert rulesAbiertoalerts bug effort:S P2
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
nightscout/nocturne#2511 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
Los mantenedores suelen responder en 2 días
-
Crash in v3.00Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
arkane-systems/mousejiggler#140 ·