QGIS plugin fails to download project due to malformed SQLite database

Open
#255 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python, sqlite
Domain
databases

Research direction

Start in mergin/client_pull.py at download_project_finalize and apply, then inspect the pygeodiff make_copy_sqlite path in mergin/deps/pygeodiff/main.py and geodifflib.py. Reproduce the production failure with the available project and full logs from @xkello. Done means the project downloads successfully, or malformed database files produce a clear actionable error.

Written by the indexing model from the issue text.

Description

bug data corruption - sync failure - crash
Description

In the QGIS plugin, a project cannot be downloaded in the production environment.

The issue happens during the project download finalization step. Based on the available log, the failure occurs while the plugin is trying to create a SQLite copy as part of the download process. The operation fails with a GeoDiff error indicating that the database disk image is malformed.

Environment

PROD

Current behavior

The project download fails in the QGIS plugin.

The available log contains a Python traceback from the download finalization flow, ending with a GeoDiff make_copy_sqlite failure caused by a malformed SQLite database.

Expected behavior

The QGIS plugin should download the project successfully, or provide a clear and actionable error message if the project contains a malformed or unreadable database file.

Additional information
  • Full logs can be provided by me: @xkello.
  • The related Odoo ticket can also be provided if needed.
Stacktrace
Traceback (most recent call last):

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/sync_dialog.py", line 151, in download_timer_tick
    download_project_finalize(self.job)

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/mergin/client_pull.py", line 247, in download_project_finalize
    task.apply(job.mp.dir, job.mp)

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/mergin/client_pull.py", line 302, in apply
    mp.geodiff.make_copy_sqlite(mp.fpath(self.file_path), mp.fpath_meta(self.file_path))

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/mergin/deps/pygeodiff/main.py", line 318, in make_copy_sqlite
    return self.clib.make_copy_sqlite(self.context, src, dst)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/mergin/deps/pygeodiff/geodifflib.py", line 476, in make_copy_sqlite
    self._parse_return_code(context, res, "make_copy_sqlite")

  File "/Users/<user>/Library/Application Support/QGIS/QGIS3/profiles/keychainone/python/plugins/Mergin/mergin/deps/pygeodiff/geodifflib.py", line 176, in _parse_return_code
    raise GeoDiffLibError(msg)

Mergin.mergin.deps.pygeodiff.geodifflib.GeoDiffLibError: Error in make_copy_sqlite:
MakeCopySqlite: backup failed: database disk image is malformed
Dominant language
Python
Stars
185
Forks
22
Avg merge
1h 31m
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MerginMaps/geodiff

All issues in MerginMaps/geodiff

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.