BulkUploadGuard's one-upload-per-user limit holds per process, not per user

Aperta
#2,578 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
42/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
postgresql, python
Ambito
api, backend, database

Direzione di ricerca

Start in api/bulk_upload_guard.py, reading the module docstring and the current per-user and global admission paths. Compare the shared-state options described in the issue and determine how the chosen approach should coordinate across mod_wsgi processes. Done means one user cannot exceed the intended limit globally while the configured global cap still behaves correctly.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

API bug

BulkUploadGuard (api/bulk_upload_guard.py) is meant to allow at most one running bulk upload per user, plus a global cap (BULK_UPLOAD_MAX_CONCURRENT, default 2). Its state is an in-memory set behind a threading.Lock, so it lives in one WSGI process. The module docstring says so ("a multi-worker deployment bounds concurrency per worker process").

What that means in production, which runs several mod_wsgi processes:

  • One user can run one upload per process, not one in total. This was reproduced on production on 2026-08-11, when the main group had 4 processes: 8 concurrent uploads by the same user gave 4 × 201 and 4 × 429. The process count has been raised since (12 in the main group when read on 2026-09-21; it is host configuration, not in this repo).
  • The global cap never binds for a single user. Each process admits up to BULK_UPLOAD_MAX_CONCURRENT uploads, and every rejection in the measurement came from the per-user branch, so for one user the effective ceiling is the process count, whatever the setting says.

The guard exists because a bulk upload pins a worker and holds an open transaction for its whole duration (synchronous WSGI by design), so the real ceiling matters for how many workers one client can occupy.

Possible fixes, roughly from cheapest:

  • a Postgres advisory lock per user (pg_try_advisory_lock(hash(user))) plus a counter for the global cap, which is shared by all processes for free;
  • a row in a small table with a TTL;
  • or accept per-process semantics and document the real ceiling (processes × 1 per user).

Related: #2362 (Bulk Upload).

Lingua principale
Python
Stelle
65
Fork
28
Merge medio
4g 7h
PR unite (30g)
25

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di OpenEnergyPlatform/oeplatform

Tutte le issue di OpenEnergyPlatform/oeplatform

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.