job table missing `dag_id` and `executor_class` for BackfillJob type
#40,055 创建于 2024年6月5日
仓库指标
- Star
- (44,809 star)
- PR 合并指标
- (平均合并 7天 18小时) (30 天内合并 834 个 PR)
描述
Apache Airflow version
2.9.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When using the airflow dags backfill command to execute a backfill for any DAG, I noticed that the dag_id and executor_class columns in the job table are empty(null).
Results in entries in the job table like this:
124460, NULL, running, BackfillJob, 2024-06-05 01:03:40.936253+00, NULL, 2024-06-05 01:22:41.634367+00, NULL, hoge-5566755f9-wbw57, airflow
However, in versions before 2.6.0, the entries in the job table included the dag_id and executor_class as expected:
2, bash_operator, running, BackfillJob, 2024-06-05 02:25:32.905082+00, NULL, 2024-06-05 02:25:48.017582+00, CeleryExecutor, a5dec317208c, airflow
I'm not sure exactly where the issue lies (or if this is a feature rather than a bug), but I will take a closer look at the code later. If I find the problem, I might submit a PR with a fix.
What you think should happen instead?
I'm not sure, but it looks like this PR https://github.com/apache/airflow/pull/30376 changed a lot in BackfillJobRunner, Maybe there is something we missed in there. Additionally, these changes are not even mentioned in the RELEASE_NOTES.rst.
reviewable stages: https://github.com/apache/airflow/pull/30255 -> https://github.com/apache/airflow/pull/30302 -> https://github.com/apache/airflow/pull/30308 -> this PR.
How to reproduce
- any simple dag file
- run airflow dags backfill with above dag's id
- check webserver's Browse > Jobs, and limit the Job Type to BackfillJob. or just check record in job table when JobType = 'BackfillJob'
This issue can be reproduced in versions from 2.6.0 to the latest version.
Operating System
MacOS 14.4.1
Versions of Apache Airflow Providers
None
Deployment
Docker-Compose
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct