Too many duplicate key value violates unique constraint errors.

未关闭
#173 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
30/100
Issue 类型
缺陷
描述清晰度
需要澄清
活跃度
停滞
技术栈
postgresql, python
领域
databases

调研方向

从 crypto/store/asyncpg/store.py 中的 put_devices 和 state_store/asyncpg/store.py 中的 set_members 开始,然后在 duplicate-key traces 中跟踪它们对 copy_records_to_table 的调用。使用多个 bot 实例重现 crypto_device 和 mx_user_profile 违规,并确定对现有行的预期处理方式。完成意味着在不丢失已存储的设备或成员关系数据的情况下处理这些反复出现的错误。

由索引模型根据 Issue 内容生成。

描述

I have logged into multiple bot accounts clients in maubot and created multiple instances for each account, relying on different plugins,
there are a lot of errors in the logs.
Although it seems that the bot is running normally, there are a lot of errors in the logs.
The logs look like this:


Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
    await handler(data)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/machine.py", line 171, in handle_device_lists
    await self._fetch_keys(device_lists.changed, include_untracked=False)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/device_lists.py", line 80, in _fetch_keys
    await self.crypto_store.put_devices(user_id, new_devices)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/store/asyncpg/store.py", line 615, in put_devices
    await conn.copy_records_to_table("crypto_device", records=data, columns=columns)
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
    return await self.wrapped.copy_records_to_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
    return await self._protocol.copy_in(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "crypto_device_pkey"
DETAIL:  Key (user_id, device_id)=(@name:domain.com, cliendid) 
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
    await handler(data)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/machine.py", line 171, in handle_device_lists
    await self._fetch_keys(device_lists.changed, include_untracked=False)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/device_lists.py", line 80, in _fetch_keys
    await self.crypto_store.put_devices(user_id, new_devices)
  File "/usr/lib/python3.11/site-packages/mautrix/crypto/store/asyncpg/store.py", line 615, in put_devices
    await conn.copy_records_to_table("crypto_device", records=data, columns=columns)
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
    return await self.wrapped.copy_records_to_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
    return await self._protocol.copy_in(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "crypto_device_pkey"
DETAIL:  Key (user_id, device_id)=(@name:domain.com, cliendid) already exists.

Sometimes very similar violates unique constraint errors appear but with different duplicate keys, such as mx_user_profile_pkey.

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/mautrix/client/syncer.py", line 236, in _catch_errors
    await handler(data)
  File "/data/plugins/org.jobmachine.chatgpt-v0.0.10.mbp/gpt.py", line 87, in on_message
    if not await self.should_respond(event):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/plugins/org.jobmachine.chatgpt-v0.0.10.mbp/gpt.py", line 67, in should_respond
    if len(await self.client.get_joined_members(event.room_id)) == 2:
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mautrix/client/store_updater.py", line 218, in get_joined_members
    await self.state_store.set_members(room_id, members, only_membership=Membership.JOIN)
  File "/usr/lib/python3.11/site-packages/mautrix/client/state_store/asyncpg/store.py", line 169, in set_members
    await conn.copy_records_to_table(
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
    return await self.wrapped.copy_records_to_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
    return await self._protocol.copy_in(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "mx_user_profile_pkey"
DETAIL:  Key (room_id, user_id)=(!xxxxxx:domain.com, @name:domain.com) already exists.

Maybe it's related to: https://github.com/mautrix/python/issues/160

主要语言
Python
星标
249
派生
84
PR 合并指标
30 天内没有已合并 PR

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

mautrix/python 的其他 Issue

查看 mautrix/python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。