help wanted
Repository-Metriken
- Stars
- (45 Stars)
- PR-Merge-Metriken
- (PR-Metriken ausstehend)
Beschreibung
Description
Tracing is a form of telemetry data which tracks fine-grain information on requests as they flow through an application.
Tracing could help us in a number of ways:
- Debugging: If someone says "my request hung," can pull a trace and immediately see: was it stuck waiting for model spin-up, or in queue, or the model crashed?
- Development Help: Can be used to study system performance before/after a new feature was added (or help debug if something new isn't working)
- Very fine-grained benchmarking: For example, determining if there any features of a pickled nnsight experiment which correlate directly with experiment length.
Overall, it provides a very clean way to visualize and study how user requests flow through our system. This could provide useful insights into how particular components of NDIF perform under varying conditions/contexts.
Desired outcome
Implement some level of cross-service tracing using opentelemetry, with span data collected in jaeger and visualized in grafana. Even just getting a basic prototype which works across our 3 services (api, queue, ray) would be an incredible milestone.