Too many duplicate key value violates unique constraint errors.
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
Research direction
Start with crypto/store/asyncpg/store.py at put_devices and state_store/asyncpg/store.py at set_members, then trace their copy_records_to_table calls in the duplicate-key traces. Reproduce the crypto_device and mx_user_profile violations with multiple bot instances and determine the expected handling for existing rows. Done means these recurring errors are handled without losing the stored device or membership data.
Written by the indexing model from the issue text.
Description
I have logged into multiple bot accounts clients in maubot and created multiple instances for each account, relying on different plugins,
there are a lot of errors in the logs.
Although it seems that the bot is running normally, there are a lot of errors in the logs.
The logs look like this:
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
await handler(data)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/machine.py", line 171, in handle_device_lists
await self._fetch_keys(device_lists.changed, include_untracked=False)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/device_lists.py", line 80, in _fetch_keys
await self.crypto_store.put_devices(user_id, new_devices)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/store/asyncpg/store.py", line 615, in put_devices
await conn.copy_records_to_table("crypto_device", records=data, columns=columns)
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
ret = await func(self, arg, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
return await self.wrapped.copy_records_to_table(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
return await self._protocol.copy_in(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "crypto_device_pkey"
DETAIL: Key (user_id, device_id)=(@name:domain.com, cliendid)
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
await handler(data)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/machine.py", line 171, in handle_device_lists
await self._fetch_keys(device_lists.changed, include_untracked=False)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/device_lists.py", line 80, in _fetch_keys
await self.crypto_store.put_devices(user_id, new_devices)
File "/usr/lib/python3.11/site-packages/mautrix/crypto/store/asyncpg/store.py", line 615, in put_devices
await conn.copy_records_to_table("crypto_device", records=data, columns=columns)
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
ret = await func(self, arg, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
return await self.wrapped.copy_records_to_table(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
return await self._protocol.copy_in(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "crypto_device_pkey"
DETAIL: Key (user_id, device_id)=(@name:domain.com, cliendid) already exists.
Sometimes very similar violates unique constraint errors appear but with different duplicate keys, such as mx_user_profile_pkey.
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
await handler(data)
File "/data/plugins/org.jobmachine.chatgpt-v0.0.10.mbp/gpt.py", line 87, in on_message
if not await self.should_respond(event):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/plugins/org.jobmachine.chatgpt-v0.0.10.mbp/gpt.py", line 67, in should_respond
if len(await self.client.get_joined_members(event.room_id)) == 2:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/mautrix/client/store_updater.py", line 218, in get_joined_members
await self.state_store.set_members(room_id, members, only_membership=Membership.JOIN)
File "/usr/lib/python3.11/site-packages/mautrix/client/state_store/asyncpg/store.py", line 169, in set_members
await conn.copy_records_to_table(
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
ret = await func(self, arg, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
return await self.wrapped.copy_records_to_table(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
return await self._protocol.copy_in(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "mx_user_profile_pkey"
DETAIL: Key (room_id, user_id)=(!xxxxxx:domain.com, @name:domain.com) already exists.
Maybe it's related to: https://github.com/mautrix/python/issues/160
- Dominant language
- Python
- Stars
- 249
- Forks
- 84
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mautrix/python
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
Similar issues
-
Add: hunch Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
AbdelStark/awesome-typesafe#104 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
DiamondLightSource/dodal#2211 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
openml/openml-python#1749 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sipyourdrink-ltd/bernstein#6191 ·