Google Vertex AI Feature Store system Dag exceeds import timeout
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- google-cloud, python
- Domain
- cloud, data-engineering
Research direction
Start with the Vertex AI Feature Store system DAG and the import behavior described in airflow-core/tests/unit/always/test_example_dags.py, running the provided Breeze pytest command first. Trace the module-scope google.cloud.aiplatform_v1beta1 and related heavy imports, then defer them until task execution. Done means the system DAG imports within the default timeout without loading that root package and passes the stated import, database-query, and connection-access checks.
Written by the indexing model from the issue text.
Description
Under which category would you file this issue?
Providers
Apache Airflow version
main (development)
What happened and how to reproduce it?
The Google provider's Vertex AI Feature Store system Dag imports google.cloud.aiplatform_v1beta1 at module scope. Importing that package triggers an expensive third-party import chain while Airflow scans the Dag file.
In a clean Breeze process, importing the unmodified system Dag took 30.505 seconds and loaded google.cloud.aiplatform_v1beta1. This exceeds Airflow's default dagbag_import_timeout of 30 seconds and can surface as a Dag import timeout or a third-party circular-import failure.
Steps to reproduce:
-
Check out
mainwith the Google provider development dependencies installed. -
Run the Vertex AI Feature Store system Dag import test in Breeze:
BACKEND=sqlite breeze run pytest airflow-core/tests/unit/always/test_example_dags.py \ -k 'test_should_be_importable and vertex_ai_feature_store' -xvs -
Observe that parsing the Dag loads
google.cloud.aiplatform_v1beta1at module scope and may exceed the 30-second Dag import timeout.
What you think should happen instead?
Scanning the system Dag should not load the Vertex AI Feature Store client package. Heavy Google clients and protobuf-related imports should be deferred until task execution so the Dag remains importable within the default timeout.
Operating System
macOS host with the Apache Airflow Breeze CI container, Python 3.10.
Deployment
Other Docker-based deployment
Apache Airflow Provider(s)
No response
Versions of Apache Airflow Providers
No response
Official Helm Chart version
Not Applicable
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
After removing the parse-time aiplatform_v1beta1 imports, the system Dag imports without loading that root package and passes Airflow's generic system Dag import, database-query, and connection-access checks.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.9k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 480
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/airflow
-
kind:bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
area:providers good first issue kind:bug kind:documentation provider:google
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
kind:bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
area:core kind:bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
area:providers kind:bug provider:cncf-kubernetes
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100