SQL Server: every stream lookup by name does a key lookup to get `Version`
Maintainers usually reply within 7 days
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
Research direction
Start with the schema in src/SqlServer/src/Eventuous.SqlServer/Scripts/1_Schema.sql, where the Streams table and UQ_StreamName are defined, and with check_stream, append_events and the CATCH block that matches the constraint name. Pull request #621 is already open against this issue, so check it before starting. Done means the lookup by name becomes a covered index seek, existing databases get an upgrade step, and the concurrency tests from #598 still pass.
Written by the indexing model from the issue text.
Description
Summary
check_stream, read_stream_forwards, and read_stream_backwards all start with the same query:
SELECT
@current_version = [Version],
@stream_id = StreamId
FROM __schema__.Streams
WHERE StreamName = @stream_name;
The only index on StreamName is the unique constraint UQ_StreamName. It holds StreamName and the clustered key StreamId, but not Version. So SQL Server seeks UQ_StreamName and then does a key lookup into PK_Streams to get Version. This occurs once for every append and every stream read.
How I found it
We capture the actual execution plans from our integration tests and analyze them with Performance Studio. It reports this statement as a critical "Key Lookup":
Key Lookup — SQL Server found rows via a nonclustered index but had to go back to the clustered index for additional columns. Columns fetched: Streams.Version
Statement in the plan (from check_stream, called by append_events):
SELECT @current_version = [Version], @stream_id = StreamId
FROM eventuous.Streams
WHERE StreamName = N'InventoryNodeStock-102'
Environment: Eventuous 0.17.0, SQL Server 2022. The schema in src/SqlServer/src/Eventuous.SqlServer/Scripts/1_Schema.sql on dev has the same index, so the plan shape is the same there.
Suggested change
Replace the unique constraint with a unique index that has the same name and includes Version:
CREATE TABLE __schema__.Streams
(
StreamId INT IDENTITY (1,1) NOT NULL,
StreamName NVARCHAR(850) NOT NULL,
[Version] INT DEFAULT (-1) NOT NULL,
CONSTRAINT PK_Streams PRIMARY KEY CLUSTERED (StreamId) WITH (OPTIMIZE_FOR_SEQUENTIAL_KEY = ON),
CONSTRAINT CK_VersionGteNegativeOne CHECK ([Version] >= -1),
INDEX UQ_StreamName UNIQUE NONCLUSTERED (StreamName) INCLUDE ([Version])
);
A UNIQUE constraint cannot have INCLUDE columns, so it must become a unique index. Keep the name UQ_StreamName: the CATCH block in check_stream looks for that name in the error message, and already handles both error 2627 (constraint) and 2601 (unique index).
Trade-offs
- Benefit: The lookup by name becomes one index seek. This saves a few logical reads on each append and each stream read, and the read path touches one index instead of two.
- Cost: The
UPDATE ... SET [Version]inappend_eventsmust now also write the nonclustered index.Versionis a 4-byteINT, so the index stays narrow, but it is one more index write for each append. - Locking:
check_streamtakesUPDLOCKon the row it reads. With a covering index, that read locks only theUQ_StreamNamekey, not the clustered row. Two concurrent appends to the same stream still meet on the sameUQ_StreamNamekey, so they must still be serialized, but the concurrency tests added in #598 should confirm this. - Existing databases:
1_Schema.sqlcreates the table onlyIF OBJECT_ID(...) IS NULL, so existing databases do not get the change. The change needs an upgrade step (drop the constraint, create the index), or a note in the release notes.
The gain for each call is small. But this query runs on the hot path of every append and every aggregate load, so I think the change is worth it. I can open a PR if you agree with the approach.
- Dominant language
- C#
- Stars
- 512
- Forks
- 99
- Avg merge
- 5d 6h
- Merged PRs (30d)
- 5
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Eventuous/eventuous
-
MongoDB integration tests fail on newer GitHub Actions runner kernelsPossibly taken @nmummau claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Eventuous/eventuous#626 · 2 comments ·
Maintainers usually reply within 7 days
-
`dotnet format style --verify-no-changes` fails with naming violations and unnecessary importsPossibly taken @nmummau claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Eventuous/eventuous#624 · 1 comment ·
Maintainers usually reply within 7 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Eventuous/eventuous#627 · 1 comment ·
Maintainers usually reply within 7 days
-
SQL Server: `append_events` does not need the `@inserted` table variablePossibly taken @nmummau claimed this 7 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 20/100
Eventuous/eventuous#619 · 1 comment ·
Maintainers usually reply within 7 days
-
SQL Server: stream reads do one key lookup per event, and `IDX_EventsStream` duplicates `UQ_StreamIdAndStreamPosition`Possibly taken @nmummau claimed this 7 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 18/100
Eventuous/eventuous#618 · 1 comment ·
Maintainers usually reply within 7 days
All issues in Eventuous/eventuous
Similar issues
-
The test-platform skill doesn't say what three of its failures look likePossibly taken @yasmoradi claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
bitfoundation/bitplatform#14022 ·
Maintainers usually reply within 1 day
-
area-TableView bug needs-triage team-Controls
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
microsoft/microsoft-ui-xaml#12351 · 1 comment ·
Maintainers usually reply within 2 days
-
AudioClient and ImageClient file path overloads send the full local path as the multipart file namePossibly taken @sebxt claimed this today. Openarea: audio area: images feature-request
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
openai/openai-dotnet#1453 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Runtime diagnostics settings report canManage=true for every callerPossibly taken A pull request linked to this issue is open or already merged. Openneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
elsa-workflows/elsa-foundation#3130 ·
Maintainers usually reply within 1 day