Improve `SELECT text` performance
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- cpp, postgresql
- Domain
- backend, databases, performance
Research direction
Start by profiling the SELECT text path and comparing it with the PostgreSQL protocol, focusing on Apache Arrow RecordBatch construction, RecordBatchWriter, and PostgreSQL's SPI_getbinval(). Determine whether copying into RecordBatch or SPI_getbinval()/nocachegetattr() dominates, then measure the completed change against the PostgreSQL protocol baseline.
Written by the indexing model from the issue text.
Description
It's slower than the PostgreSQL protocol.
The followings may be related but we need to look into it:
- Building
arrow::RecordBatchis slow?- We need to build
arrow::RecordBatches to usearrow::ipc::RecordBatchWriter(). We need to copy PostgreSQL data for it. (Not zero-copy.) - Should we add an API that writes Apache Arrow streaming format data without building
arrow::RecordBatchto Apache Arrow C++?
- We need to build
- Calling
SPI_getbinval()is slow?- It calls
nocachegetattr()https://github.com/postgres/postgres/blob/3edc6580c0e27fb8f13322efd255a88d20dda6c2/src/backend/access/common/heaptuple.c#L496-L712 and it's not a short function. Can we shortcut some operations?
- It calls
- Dominant language
- C++
- Stars
- 116
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-flight-sql-postgresql
-
Difficulty 2/5 1-3 hours Newbie friendliness 30/100
apache/arrow-flight-sql-postgresql#190 · 26 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 38/100
apache/arrow-flight-sql-postgresql#189 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 35/100
apache/arrow-flight-sql-postgresql#185 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Improve timeout case Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
All issues in apache/arrow-flight-sql-postgresql
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
duckdb/duckdb-wasm#2258 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
objectionary/eo-graphs#75 ·
-
Coarray integration tests carry no LABELS, so run_tests.py silently skips them under every backend Opencoarray
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
FISCO-BCOS/FISCO-BCOS#5642 ·