Subscription gap metric reads the end of the same stream once for each subscription
メンテナーはふだん 7 日以内に返信
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 35/100
- issue の種類
- 機能追加
- 明瞭さ
- 明確に書かれている
- 活発さ
- 停滞
- 技術スタック
- csharp
調査の方向性
Start with src/Core/src/Eventuous.Subscriptions/Diagnostics/SubscriptionMetrics.cs, focusing on ObserveGapValues and GetGap, then trace how measures are registered in SubscriptionRegistrationExtensions.cs. Review the SQL and KurrentDB measure implementations listed in the issue to see how they identify an end-of-stream source. Done means each shared source is read once per collection while subscription-id tags and custom implementations without a key retain their stated behavior; linked pull request 615 is already open.
索引モデルが issue の本文から書いたものです。
説明
Is your feature request related to a problem? Please describe.
AddEventuousSubscriptions() registers the subscription gap gauges in SubscriptionMetrics. On each metric collection, ObserveGapValues calls the GetSubscriptionEndOfStream delegate of every measured subscription, one after another. Each delegate reads the end of its stream from the store.
When several subscriptions read the same source, every call returns the same value, but the store gets one request for each subscription. This happens in the core SubscriptionMetrics, so it affects every store that supports the gap metric:
| Store | End-of-stream request for an all-stream subscription |
|---|---|
| SQL Server | SELECT MAX(GlobalPosition) FROM <schema>.Messages |
| PostgreSQL | select max(global_position) from <schema>.messages |
| SQLite | SELECT MAX(global_position) FROM <messages table> |
| KurrentDB | ReadAllAsync(Direction.Backwards, Position.End, 1), a gRPC read of the last event in $all |
For example, an application has six all-stream subscriptions on one SQL Server store. Each collection then sends the same query six times. The position column is the clustered primary key, so each SQL query is cheap. For KurrentDB, each call is a network round trip. In all cases, the number of requests grows with the number of subscriptions and with the export frequency:
| Metric export interval | Subscriptions | End-of-stream requests |
|---|---|---|
| 60 seconds | 6 | 6 each minute |
| 200 milliseconds | 6 | about 30 each second |
A second, related problem: GetGap runs each delegate synchronously (GetAwaiter().GetResult()) on the collection thread, with a 500 ms timeout for each subscription. When the store is slow, one collection can block for up to 500 ms multiplied by the number of subscriptions. This also applies to every store.
Observed with Eventuous 0.17.0 (Eventuous.SqlServer, Eventuous.Diagnostics.OpenTelemetry). The other stores are from a reading of the dev branch source.
Describe the solution you'd like
Read each end-of-stream source one time per collection, and share the result between the subscriptions that read that source. Make the change in SubscriptionMetrics, so that it applies to all stores.
A possible shape:
IMeasuredSubscriptionexposes a key for its end-of-stream source in addition to the delegate. Subscriptions that read the same source return the same key. For example, a SQL all-stream subscription returns the store type, the connection, and the schema. A stream subscription adds the stream name. A KurrentDB$allsubscription returns the client and$all.SubscriptionMetricsgroups the measures by key. On each collection, it calls one delegate for each key and uses the result for every subscription in the group. Thesubscription-idtag stays the same, so the exported series do not change.- Optionally,
SubscriptionMetricsruns the remaining calls concurrently, with one overall timeout, rather than one blocking call after another.
The key could be optional. A subscription with no key keeps the current behavior, so custom IMeasuredSubscription implementations continue to work.
Additional context
The relevant code:
src/Core/src/Eventuous.Subscriptions/Diagnostics/SubscriptionMetrics.cs:ObserveGapValuesandGetGap.src/Core/src/Eventuous.Subscriptions/Registrations/SubscriptionRegistrationExtensions.cs: registers oneGetSubscriptionEndOfStreamfor each measured subscription.src/Relational/src/Eventuous.Sql.Base/Subscriptions/SqlSubscriptionBase.cs:GetSubscriptionEndOfStream, which SQL Server, PostgreSQL, and SQLite share.PrepareEndOfStreamCommandin the all-stream and stream subscriptions ofEventuous.SqlServer,Eventuous.Postgresql, andEventuous.Sqlite.src/KurrentDB/src/Eventuous.KurrentDB/Subscriptions/Diagnostics/:AllStreamSubscriptionMeasure,StreamSubscriptionMeasure, andBaseSubscriptionMeasure.
- 主要言語
- C#
- スター
- 512
- フォーク
- 99
- 平均マージ
- 5日 6時間
- マージ済み PR(30日)
- 5
環境構築
- Dockerfile または Docker Compose ファイルあり
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
Eventuous/eventuous のほかの issue
-
MongoDB integration tests fail on newer GitHub Actions runner kernels対応中かも @nmummau が 3 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
Eventuous/eventuous#626 · コメント 2 件 ·
メンテナーはふだん 7 日以内に返信
-
`dotnet format style --verify-no-changes` fails with naming violations and unnecessary imports対応中かも @nmummau が 3 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100
Eventuous/eventuous#624 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
Eventuous/eventuous#627 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
-
SQL Server: `append_events` does not need the `@inserted` table variable対応中かも @nmummau が 9 日前に担当しました。 オープン
難易度 3/5 1〜2日 初心者へのやさしさ 20/100
Eventuous/eventuous#619 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
-
SQL Server: stream reads do one key lookup per event, and `IDX_EventsStream` duplicates `UQ_StreamIdAndStreamPosition`対応中かも @nmummau が 9 日前に担当しました。 オープン
難易度 3/5 1〜2日 初心者へのやさしさ 18/100
Eventuous/eventuous#618 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
Eventuous/eventuous の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
Nimblesite/SharpLsp#468 ·
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
elastic/ecs-dotnet#575 ·
-
alerts bug effort:S P2
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
nightscout/nocturne#2511 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 2 日以内に返信
-
Crash in v3.00オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 62/100
arkane-systems/mousejiggler#140 ·