Regression in copy records performance between 0.30.0 and 0.31.0

Open
#1,327 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
postgresql, python
Domain
databases, performance

Research direction

Start by running the supplied asyncio script with asyncpg 0.30.0 and 0.31.0 against PostgreSQL, confirming the reported copy_records_to_table timing difference. Trace the copy_records_to_table path to identify what changed between those versions; done means the regression is explained and the benchmark no longer shows the reported slowdown.

Written by the indexing model from the issue text.

Description

When run on different versions (0.30.0 vs 0.31.0), the following script reproduces the issue and consistently shows a performance regression of approximately 22%:

import asyncio
import asyncpg
import time

COUNT = 5_000

async def run():
    conn = await asyncpg.connect("postgresql://user:password@localhost/postgres")

    await conn.execute("CREATE TABLE IF NOT EXISTS users (id SERIAL PRIMARY KEY, username TEXT, email TEXT, age INT)")

    RECORDS = [("John Doe", "john.doe@example.com", 42)] * 10_000

    start = time.perf_counter()
    for _ in range(COUNT):
        await conn.copy_records_to_table("users",  records=RECORDS,  columns=["username", "email", "age"])
    end = time.perf_counter()

    await conn.close()

    return end - start


if __name__ == "__main__":
    total = asyncio.run(run())
    print(f"Total time: {total:.5f} seconds")
    print(f"Average time per loop: {total / COUNT:.8f} seconds")

(common) [e.lee@OC0000842]$ pip install asyncpg==0.30.0
(common) [e.lee@OC0000842]$ python -m example
Total time: 89.15468 seconds
Average time per loop: 0.01783094 seconds

(common) [e.lee@OC0000842]$ pip install asyncpg==0.31.0
(common) [e.lee@OC0000842]$ python -m example
Total time: 108.57080 seconds
Average time per loop: 0.02171416 seconds
Dominant language
Python
Stars
8.1k
Forks
469
Avg merge
4h 42m
Merged PRs (30d)
6

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MagicStack/asyncpg

All issues in MagicStack/asyncpg

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.