Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

perf(results): build typed columns directly from the driver stream instead of collecting List<List<Object?>> first

Offen
  • Stimmt etwas nicht?
#1,552 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
42/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
dart, mysql, postgresql, sqlite
Bereich
database, performance

Rechercherichtung

Read postgres_sql_workspace.dart, mysql_sql_workspace.dart, and sqlite_sql_workspace.dart to trace how each driver currently collects rows and builds CompactResultDataset; then inspect CompactResultDataset.fromRawRows and the related work in #1548. Define a shared builder that preserves current mixed-type and NULL behavior, and use existing dataset tests to check it across all three drivers. Done means those tests pass and the PR reports peak RSS and timings for 5k and 50k rows.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

backend core enhancement P3 performance
Problem

Each SQL driver workspace builds an intermediate List<List<Object?>> of the whole result and only then packs it into typed columns (CompactResultDataset.fromRawRows):

  • PostgreSQL (postgres_sql_workspace.dart): rawRows.add(row.toList()) for every row.
  • MySQL (mysql_sql_workspace.dart): taken.items (all ResultSetRows kept) and then [for (i...) row.colAt(i)] per row.
  • SQLite: same pattern (sqlite_sql_workspace.dart).

So every cell is copied into a per-row list once, kept alive until the dataset is built, and then read again column by column.

Measurements

Local PostgreSQL 16 / MySQL 8.4 containers, 100k rows x 20 columns, dart run, median of 3:

Driver Rows Network + decode Extra row copy Longest event-loop stall during it
PostgreSQL 5,000 21-22 ms 1-3 ms (row.toList) 0 ms
PostgreSQL 50,000 257-271 ms 10-19 ms 0-2 ms
MySQL 5,000 31-38 ms 2-5 ms (colAt) 4-7 ms
MySQL 50,000 296-320 ms 33-42 ms 7-15 ms

(The decode itself yields between socket chunks: the PostgreSQL driver blocks the event loop for 0-2 ms even at 50k rows, so moving the driver into a worker isolate would not remove a visible hitch; it only adds connect cost and a result hand-off. Not recommended.)

Solution

Let the driver loop append into column builders (int / double / string / generic with a null bitmap) as rows arrive, deciding the kind from the first rows and falling back to generic when a value does not fit, instead of collecting List<List<Object?>>. A small ResultColumnsBuilder would be shared by the three drivers and by #1548's sampled detection.

Expected: removes the 1-3 ms (5k) / 10-40 ms (50k) copy and the peak memory of the intermediate list, and makes the "decode / pack" step incremental.

Acceptance
  • A shared builder used by PostgreSQL, MySQL and SQLite workspaces; existing dataset tests unchanged.
  • Mixed-type and NULL-heavy columns still end up as generic / string exactly as today (tests).
  • Peak RSS and time for 5k and 50k rows reported in the PR.

Related: #1548, #1549.

Vorherrschende Sprache
Dart
Sterne
23
Forks
1
Ø Merge
31 Min.
Gemergte PRs (30 T.)
570

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus QueryaHub/Querya-Desktop

Alle Issues in QueryaHub/Querya-Desktop

Ähnliche Issues

Weitere Issues zu Dart

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.