Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Bug]: Laravel endpoint collection fails with EMSGSIZE when the app has ~3,000 routes and is retried on every request

Abierto
  • ¿Algo no está bien?
#4,371 5 comentarios 1 reacción 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
laravel, linux, php
Área
backend, distributed-systems

Línea de trabajo

Start with the LaravelIntegration hook around Router::findRoute and the add_endpoint/flush_endpoints calls described in the issue, then trace how libdd-ipc sets max_message_size() and how blocking::enqueue_actions sends EnqueueActions requests. Reproduce with a route batch large enough to exceed the effective SOCK_SEQPACKET limit. Done means endpoint collection succeeds for large route sets without retrying the full collection on each request.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Bug report

Summary

In the Laravel integration, endpoint collection (add_endpoint + flush_endpoints) sends every route to the sidecar in a single IPC message. With enough routes the message exceeds the AF_UNIX SOCK_SEQPACKET limit and sendmsg fails with EMSGSIZE. The error is swallowed, are_endpoints_collected() never becomes true, and the whole collection is repeated on every request.

Environment

  • dd-trace-php 1.24.2, 1.25.1, 1.26.0, 1.26.1 and 1.26.2 (same behavior in all of them)
  • PHP 8.3 (NTS), php-fpm + Apache, Linux, Laravel 11 with route:cache, AppSec and profiling installed (AppSec enabled)
  • ~3,000 routes (php artisan route:list --json | jq length)

What happens

LaravelIntegration (Router::findRoute hook) does, on each request while !\DDTrace\are_endpoints_collected():

foreach ($router->getRoutes() as $route) { \DDTrace\add_endpoint(...); }
\DDTrace\flush_endpoints();

strace -f -e trace=sendmsg on a php-fpm worker shows the batch going out as one message that fails:

sendmsg(8, {msg_iov=[{iov_base="...", iov_len=427136}], ...}, 0) = -1 EMSGSIZE (Message too long)

The batch is ~53 bytes per route plus the strings (3,037 routes ≈ 427 KB). The socket buffer is requested as 4 MiB, but the kernel caps SOCK_SEQPACKET messages at about 2 x net.core.wmem_max (212,992 by default, so ~425,984). net.core.wmem_max is not tunable per container (the file does not exist in the container netns), so containers cannot raise the limit.

With the batch just below the limit (e.g. dropping ~20 routes), collection succeeds and are_endpoints_collected() becomes true, so the problem appears suddenly when an app crosses ~3,000 routes.

Impact

  • The collection is rebuilt on every request (CompiledRouteCollection::getRoutes() -> ~3,000 newRoute), about +0.8 vCPU per busy worker pool in our measurements.
  • The sidecar transport is closed and reconnected on each request (The sidecar transport is closed. Reconnecting...) and AppSec logs PHP Warning: [ddappsec] Error in message exchange for command request_shutdown: dd_helper_fatal on every request.
  • In our production deployment, tasks that crossed the limit started writing to disk at ~100 MB/s about 1-2 minutes after receiving traffic and filled their 20 GiB ephemeral storage in 2-3 minutes. We could not conclusively identify the writing process (the sidecar log files are regular files named pipe:[inode] created in the working directory when error_log=/dev/stderr), so we are not claiming the sidecar is the writer; the strong correlation is with this failure loop. The same image on a staging with ~200x less traffic did not fill the disk.

Code reading (master / main)

  • libdd-ipc requests a 4 MiB socket buffer (SOCKET_BUFFER_SIZE) and uses it as max_message_size(), but the effective Linux limit is lower.
  • blocking::enqueue_actions sends all actions in one EnqueueActions request and ignores the result (let _ = transport.with_retry(...)).
  • No config option (DD_API_SECURITY_..., appsec, telemetry) disables endpoint collection in the Laravel integration; DD_INSTRUMENTATION_TELEMETRY_ENABLED=false, DD_APPSEC_ENABLED=false, DD_TRACE_SIDECAR_CONNECTION_MODE=thread and DD_TRACE_SIDECAR_TRACE_SENDER=false individually do not avoid the EMSGSIZE.

Expected behavior

Endpoint collection should not fail or loop when an application has many routes. Either split the batch into several messages below the effective limit, or at least stop retrying on every request when the send fails, and expose a setting to disable the collection.

Workaround

A global middleware that calls add_endpoint and flushes every ~150 KB while !are_endpoints_collected() avoids the error (3 messages of ~146 KB for 3,037 routes; are_endpoints_collected() becomes true after the first message and the agent receives all 3,037 endpoints in app-endpoints).

Related

This is the same pattern as #3760 (WordPress integration collecting endpoints on every request until are_endpoints_collected() is true), which was resolved in 1.17.1 by disabling the collection for WordPress.

Lenguaje dominante
PHP
Estrellas
558
Forks
192
Merge medio
2 d 3 h
PR fusionados (30 d)
104

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de DataDog/dd-trace-php

Todos los issues de DataDog/dd-trace-php

Issues similares

Más issues de PHP

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.