Track /bulk-lookup query times in /status, e.g. as recent_bulk_queries
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Necesita aclaración
- Estado de actividad
- Activo
- Stack tecnológico
- kubernetes, python
- Área
- api, backend, observability
Línea de trabajo
Read api/server.py, especially lookup() and RecentLookups, and the recent_queries section of documentation/API.md; compare the existing /lookup tracking with bulk_lookup(). The issue leaves the entry unit, reported metrics, warning thresholds, and tracker design open. Done means those choices are settled and /status provides useful bulk latency and failure visibility without distorting /lookup statistics or adding undue probe cost.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
/status's recent_queries times /lookup only. #231 deliberately leaves /bulk-lookup out: bulk_lookup() calls lookup(..., bulk=True) for each string, and those calls are neither recorded in the window nor logged as SLOW QUERY warnings. That keeps recent_queries meaning what its docs say. A single bulk request can hold thousands of strings, enough to swamp a 50000-entry window, and its strings compete with each other for Solr in a way a /lookup caller's doesn't, which would skew the percentiles. (See the bulk parameter's docstring on lookup() in api/server.py, and the recent_queries section of documentation/API.md.)
That leaves no latency or failure visibility for bulk traffic beyond the one Bulk lookup query for N strings ... took Xms INFO line per request. Bulk is the path NER pipelines use, so it is likely where most Solr load comes from.
One option is a separate tracker, e.g. recent_bulk_queries alongside recent_queries in /status. Questions to settle:
- What is one entry: a request or a string? Per request is closest to what a caller sees, but the times are only comparable after normalising by string count. Per string is comparable with
recent_queries, but needs a window sized differently, or one bulk request still dominates it. - What to report. Candidates: requests, strings per request, end-to-end time per request, per-string mean and p99 Solr wait, time spent waiting on the
SOLR_MAX_CONCURRENT_LOOKUPSsemaphore (whichlookup()'s own timer doesn't include), anderrors/timeouts/cancelledas inrecent_queries. - Logging. Should a slow bulk request (by some per-string or total threshold) get its own WARNING, instead of the per-string
SLOW QUERYlines that were dropped? - Cost.
/statusis what the Kubernetes probes call.RecentLookupsinapi/server.pykeeps running totals and caches percentiles for this reason, and a second tracker should do the same, or reuse that class.
Related: #286 (query-rate windows for recent_queries).
- Lenguaje dominante
- Python
- Estrellas
- 5
- Forks
- 4
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Sin plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de NCATSTranslator/NameResolution
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
-
Dificultad 4/5 3-5 días Aptitud para principiantes 43/100
-
Dificultad 3/5 1-2 días Aptitud para principiantes 66/100
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 25/100
-
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
Todos los issues de NCATSTranslator/NameResolution
Issues similares
-
cognition facet:consolidation memory ready size:S
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día
-
proposal
Dificultad 1/5 Menos de una hora Aptitud para principiantes 68/100
j3ko/agent-plugins#29 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
-
area:cli area:security bug priority:medium
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
rtk-ai/rtk#5127 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 74/100
Gathin23/cleanyourtv#4026 ·
Los mantenedores suelen responder en 1 día