help wanted
Métriques du dépôt
- Stars
- (45 stars)
- Métriques de merge PR
- (Métriques PR en attente)
Description
Description
Tracing is a form of telemetry data which tracks fine-grain information on requests as they flow through an application.
Tracing could help us in a number of ways:
- Debugging: If someone says "my request hung," can pull a trace and immediately see: was it stuck waiting for model spin-up, or in queue, or the model crashed?
- Development Help: Can be used to study system performance before/after a new feature was added (or help debug if something new isn't working)
- Very fine-grained benchmarking: For example, determining if there any features of a pickled nnsight experiment which correlate directly with experiment length.
Overall, it provides a very clean way to visualize and study how user requests flow through our system. This could provide useful insights into how particular components of NDIF perform under varying conditions/contexts.
Desired outcome
Implement some level of cross-service tracing using opentelemetry, with span data collected in jaeger and visualized in grafana. Even just getting a basic prototype which works across our 3 services (api, queue, ray) would be an incredible milestone.