nsqio/nsq

nsqd: record delivery to FIN metrics

Open

#347 opened on May 19, 2014

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Go (25,788 stars) (2,898 forks)batch import
featurehelp wanted

Description

We currently have end-to-end (from PUB to FIN) metrics in NSQ. We need another distinct set of "e2e" metrics that measure delivery (the point in time nsqd wrote the message to the wire) to FIN.

This way you could separate the overall end-to-end processing health from the current health of the consumption side.

Obviously, when the queue is 30mm behind the existing E2E metrics are gonna go through the roof (and from a business perspective it's a good metric to know).

But, for debugging, you want to be able to understand how fast your workers are currently able to process messages. This would help identify when it's a consumer/downstream problem vs. something else. i.e. If it normally takes 2seconds to process messages but now it's taking 10seconds, that would help explain why you have a backlog.

cc @elubow

Contributor guide

Tech stack
go
Domain
backendobservability
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go programmingNSQ messaging conceptsMetrics instrumentation
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45
Research direction
Examine the existing end to end metric implementation in the nsqd package, likely in code related to message handling and stats. Identify where the PUB time is captured and where FIN is acknowledged. Add a new metric that records the timestamp when the message is written to the wire (delivery) and compute the difference when FIN is received. Reference the conversation in the issue, including the comment by @elubow, to ensure the solution matches the intended behavior. Consider using the same instrumentation library already in use.