Metric flush cannot be triggered automatically

Abierto
  • ¿Algo no está bien?
#1,949 1 comentario 2 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Estancado
Stack tecnológico
aws, nodejs
Área
cloud, observability-sre

Línea de trabajo

Start with the linked otlp-lambda-example reproduction and its docs/test-results.md automatic-flush section, using the Node.js 20 and listed layer versions. Trace the automatic flush and shutdown behavior around the HTTP exporter timeout and repeated Grafana emissions. Done means the root cause is identified and the expected metric export completes without timeout or duplicate emissions.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug javascript

Describe the bug
I noticed that automatic flushing is triggered before lambda receives shutdown signal. However, it makes the http exporter fail with http timeout.

Forcing the metric flush or triggering shutdown sequence during lambda invocation works. In this case, I've observed same metric being emitted to grafana 5 times, each 1 minute apart. This part of the bug may not be relevant to the project; but I'm unable to find the root cause.

Steps to reproduce
I've created this repo that simulates the behavior. You can find the details logs of the test results there as well.

What did you expect to see?
Basic config of otel exporter with http exproter successfully exports metrics.

What did you see instead?

ERROR	{"stack":"Error: PeriodicExportingMetricReader: metrics export failed (error Error: Request Timeout)\n    at d._doRun (/opt/773.wrapper.js:1:3565)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at runNextTicks (node:internal/process/task_queues:64:3)\n    at process.processTimers (node:internal/timers:516:9)\n    at async d._runOnce (/opt/773.wrapper.js:1:2894)\n    at async d.onForceFlush (/opt/773.wrapper.js:1:3784)\n    at async d.forceFlush (/opt/773.wrapper.js:1:2000)\n    at async ee.forceFlush (/opt/773.wrapper.js:1:21040)\n    at async Promise.all (index 0)\n    at async ne.forceFlush (/opt/773.wrapper.js:1:24412)","message":"PeriodicExportingMetricReader: metrics export failed (error Error: Request Timeout)","name":"Error"}

What version of collector/language SDK version did you use?
node20
layers:

  • opentelemetry-nodejs-0_16_0:1
  • opentelemetry-collector-amd64-0_17_0:1

What language layer did you use?
nodejs

Additional context

Reproducible version can be found in https://github.com/kaskavalci/otlp-lambda-example/tree/main

It's weird that when shutdown or forceflush is triggered, same metric appears 5 times in grafana. I appreaciate your comment on this as well.

Lenguaje dominante
Go
Estrellas
433
Forks
249
Merge medio
1 d 12 h
PR fusionados (30 d)
39

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de open-telemetry/opentelemetry-lambda

Todos los issues de open-telemetry/opentelemetry-lambda

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.