apache/airflow

skip_archive should actually skip archive in db clean command

Open

#42,003 opened on Sep 4, 2024

View on GitHub
 (8 comments) (1 reaction) (0 assignees)Python (16,781 forks)batch import
area:db-migrationsgood first issuekind:featurekind:meta

Repository metrics

Stars
 (44,809 stars)
PR merge metrics
 (Avg merge 7d 18h) (834 merged PRs in 30d)

Description

Body

Currently the delete process uses the archive table in the process of deletion. But we should refactor this so it doesn't need this and actually not create the archive table at all when user specifies this option.

The issue is when you are deleting a lot of rows, particularly if you have a low statement timeout, you may hit errors.

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Research direction
Inspect the db clean command implementation and trace the archive table usage.
Tech stack
python
Domain
backenddatabase
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75