dotnet/orleans

[PERF] Investigate optimizations for large graphs

Open

#8,140 opened on Nov 12, 2022

View on GitHub
 (1 comment) (1 reaction) (0 assignees)C# (10,777 stars) (2,123 forks)batch import
area-performancearea-serializationenhancementhelp wanted

Description

See this twitter thread for reference: https://twitter.com/neuecc/status/1589902671540256771?s=20&t=AcVRk8mwSdMyQJT2c5jyDg

Note: we could also consider avoiding reference tracking for types by default. There are some complexities involved, like what do you do with built-in types, eg string, byte[], List<object>. You could potentially have ref-tracking variants of the codecs or pass a reference tracking preference to the codec and have an attribute per-property to indicate a reference tracking preference.

Contributor guide

Tech stack
csharp
Domain
performancebackend
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
C# knowledgeOrleans framework basicsserialization conceptsperformance profiling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
Investigate the referenced Twitter thread by neuecc for optimizations used in C# serialization, particularly for large graphs. Consider avoiding reference tracking by default for types like string, byte[], List<object>. Explore creating ref tracking variants of codecs or passing a reference tracking preference to the codec with an attribute per property. Examine existing Orleans serialization code, likely in the Orleans.Serialization namespace, to identify current implementation details and potential optimization points.