Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

perf(results): build typed columns directly from the driver stream instead of collecting List<List<Object?>> first

Aberta
  • Algo errado?
#1,552 2 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
42/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
dart, mysql, postgresql, sqlite
Domínio
database, performance

Direção de pesquisa

Read postgres_sql_workspace.dart, mysql_sql_workspace.dart, and sqlite_sql_workspace.dart to trace how each driver currently collects rows and builds CompactResultDataset; then inspect CompactResultDataset.fromRawRows and the related work in #1548. Define a shared builder that preserves current mixed-type and NULL behavior, and use existing dataset tests to check it across all three drivers. Done means those tests pass and the PR reports peak RSS and timings for 5k and 50k rows.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

backend core enhancement P3 performance
Problem

Each SQL driver workspace builds an intermediate List<List<Object?>> of the whole result and only then packs it into typed columns (CompactResultDataset.fromRawRows):

  • PostgreSQL (postgres_sql_workspace.dart): rawRows.add(row.toList()) for every row.
  • MySQL (mysql_sql_workspace.dart): taken.items (all ResultSetRows kept) and then [for (i...) row.colAt(i)] per row.
  • SQLite: same pattern (sqlite_sql_workspace.dart).

So every cell is copied into a per-row list once, kept alive until the dataset is built, and then read again column by column.

Measurements

Local PostgreSQL 16 / MySQL 8.4 containers, 100k rows x 20 columns, dart run, median of 3:

Driver Rows Network + decode Extra row copy Longest event-loop stall during it
PostgreSQL 5,000 21-22 ms 1-3 ms (row.toList) 0 ms
PostgreSQL 50,000 257-271 ms 10-19 ms 0-2 ms
MySQL 5,000 31-38 ms 2-5 ms (colAt) 4-7 ms
MySQL 50,000 296-320 ms 33-42 ms 7-15 ms

(The decode itself yields between socket chunks: the PostgreSQL driver blocks the event loop for 0-2 ms even at 50k rows, so moving the driver into a worker isolate would not remove a visible hitch; it only adds connect cost and a result hand-off. Not recommended.)

Solution

Let the driver loop append into column builders (int / double / string / generic with a null bitmap) as rows arrive, deciding the kind from the first rows and falling back to generic when a value does not fit, instead of collecting List<List<Object?>>. A small ResultColumnsBuilder would be shared by the three drivers and by #1548's sampled detection.

Expected: removes the 1-3 ms (5k) / 10-40 ms (50k) copy and the peak memory of the intermediate list, and makes the "decode / pack" step incremental.

Acceptance
  • A shared builder used by PostgreSQL, MySQL and SQLite workspaces; existing dataset tests unchanged.
  • Mixed-type and NULL-heavy columns still end up as generic / string exactly as today (tests).
  • Peak RSS and time for 5k and 50k rows reported in the PR.

Related: #1548, #1549.

Linguagem predominante
Dart
Estrelas
23
Forks
1
Merge médio
31min
PRs com merge (30d)
570

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de QueryaHub/Querya-Desktop

Todas as issues de QueryaHub/Querya-Desktop

Issues semelhantes

Mais issues de Dart

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.