Bound the total number of Solr queries the API has in flight across concurrent requests

Abierto
#311 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
python

Línea de trabajo

Start with api/server.py and Config.solr_max_concurrent_lookups, then review the shared httpx client and connection limits introduced by #310. Use the Solr capacity benchmark in TranslatorSRI/babel-validation#107 and the timing work in #309 to guide decisions about the process-wide limit, fairness, and overload behavior. Done means Solr concurrency is bounded across endpoints and concurrent requests, with queueing or rejection behavior and its timeout outcomes covered.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Nothing bounds how many Solr queries the NameRes API process has in flight in total.

  • SOLR_MAX_CONCURRENT_LOOKUPS (default 100) is a semaphore per /bulk-lookup request (Config.solr_max_concurrent_lookups in api/server.py). Its comment notes that the service-wide total is that limit multiplied by the number of bulk requests being served at once, and that the default of 100 "should be revisited once we know the real request rate", pointing at TranslatorSRI/babel-validation#107.
  • /lookup, /synonyms and the rest have no limit at all.
  • #310 replaces the per-query httpx clients with one shared client per event loop. It deliberately leaves that client's pool uncapped (httpx.Limits(max_connections=None, ...)) to keep today's behaviour. A cap below what /bulk-lookup can have in flight would make queries wait for a pool slot inside SOLR_TIMEOUT_SECONDS, so a busy pool would show up as Solr timeouts.

So a few concurrent large bulk requests, or a burst of autocomplete traffic, can put hundreds of queries on Solr at once. #306 (six concurrent lookups for "vitamin a", each blocked on Solr for 4-5 minutes) may be this or something else, but nothing in the API stops a pile-up like it.

Questions to settle:

  • Where the bound lives. A process-wide semaphore around every Solr request, a capped httpx pool, or both. With a capped pool, the wait for a connection needs its own timeout (httpx's pool timeout), separate from Solr's, so that recent_queries can tell "NameRes queued it" apart from "Solr was slow". That probably means another outcome alongside timeouts.
  • Fairness. Whether one /bulk-lookup can take every slot, starving /lookup (interactive autocomplete), or whether bulk gets a share.
  • The number. It should come from Solr's capacity under the measured request mix (babel-validation#107's benchmark), not a guess. It also needs dividing by the number of API replicas, since each replica has its own limit.
  • What callers see when it's full: queueing, or a fast 503 with Retry-After.

Related: #309 (timing /bulk-lookup, which would show how long bulk strings wait for a slot).

Lenguaje dominante
Python
Estrellas
5
Forks
4
Métricas de merge de PR
Sin PR fusionados en 30 d

Preparar el entorno

  • Incluye un Dockerfile o un archivo de Docker Compose
  • Sin plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NCATSTranslator/NameResolution

Todos los issues de NCATSTranslator/NameResolution

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.