DAG processor treats any zip-format file (.jar, .pptx, .docx, .xlsx, etc.) as a potential DAG bundle, not just .zip

Open Beginner friendly
#71,125 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python

Research direction

Start in airflow-core/src/airflow/utils/file.py at find_dag_file_paths and trace how might_contain_dag is reached. Reproduce with a non-.zip PK-format file and a .zip DAG bundle, then verify that only Python files and supported ZIP bundles are considered for DAG discovery, while unrelated archive-based formats are skipped.

Written by the indexing model from the issue text.

Description

affected_version:3.3 area:core area:dag-bundles area:DAG-processing kind:bug
Description

find_dag_file_paths (airflow-core/src/airflow/utils/file.py) decides whether to attempt DAG discovery on a file using:

if path.is_file() and (path.suffix == ".py" or zipfile.is_zipfile(path)):
    if might_contain_dag(file_path, safe_mode):
        file_paths.append(file_path)

zipfile.is_zipfile() is a content sniff (checks for the PK zip magic bytes/central directory), not an extension check. There is no accompanying path.suffix == ".zip" condition. Since the ZIP format underlies many common file types beyond .zip itself — .jar, .pptx, .docx, .xlsx, .apk, .epub, .odt, .whl, etc. — any of these dropped into a DAGs folder (e.g. a build artifact, a supporting doc, a packaged dependency) will pass this check and get opened and scanned via might_contain_dag, purely because it happens to share the underlying zip container format with Airflow's own zipped-DAG-bundle feature.

This was reported previously in #45718 with a .pptx file, but that issue was closed as invalid because the specific symptom described there (garbled metric names) turned out to be an unrelated stat-sanitization bug, not this zip-detection design question. The zip-detection behavior itself was never actually addressed.

Impact

At minimum this is wasted work (opening and scanning irrelevant files on every DAG processor cycle). Depending on might_contain_dag's heuristic and the archive's contents, it can also produce confusing log noise, and in the originally reported case, appears able to feed corrupted/unexpected data further into DAG processing.

What you think should happen instead

Gate the zip-bundle branch on the file extension in addition to (or instead of) the content sniff, e.g.:

if path.is_file() and (path.suffix == ".py" or (path.suffix == ".zip" and zipfile.is_zipfile(path))):

This preserves the documented .zip DAG-bundle behavior while no longer opening arbitrary non-.zip files that merely share the same underlying container format. .airflowignore is a viable per-deployment workaround today (matching by extension), but it means every deployment that happens to keep e.g. .jar files anywhere under its DAGs folder has to know to add this rule proactively rather than it being a non-issue by default.

How to reproduce
  1. Place any non-.zip PK-zip-format file (a .jar, .pptx, .docx, etc.) anywhere under the DAGs folder.
  2. Wait for the DAG processor to walk the directory.
  3. Observe that the file is opened and passed through might_contain_dag (visible via DAG processor debug logs), the same as a .zip DAG bundle would be, despite not being one.
Are you willing to submit a PR?
  • Yes
Code of Conduct
  • I agree to follow this project's Code of Conduct

Drafted-by: Claude Code (Sonnet 5); reviewed by @seanmuth before posting

Dominant language
Python
Stars
46.9k
Forks
17.9k
Avg merge
2d 5h
Merged PRs (30d)
480

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/airflow

All issues in apache/airflow

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.