Google Vertex AI Feature Store system Dag exceeds import timeout

Open Beginner friendly
#69,822 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
google-cloud, python

Research direction

Start with the Vertex AI Feature Store system DAG and the import behavior described in airflow-core/tests/unit/always/test_example_dags.py, running the provided Breeze pytest command first. Trace the module-scope google.cloud.aiplatform_v1beta1 and related heavy imports, then defer them until task execution. Done means the system DAG imports within the default timeout without loading that root package and passes the stated import, database-query, and connection-access checks.

Written by the indexing model from the issue text.

Description

area:providers kind:bug needs-triage provider:google
Under which category would you file this issue?

Providers

Apache Airflow version

main (development)

What happened and how to reproduce it?

The Google provider's Vertex AI Feature Store system Dag imports google.cloud.aiplatform_v1beta1 at module scope. Importing that package triggers an expensive third-party import chain while Airflow scans the Dag file.

In a clean Breeze process, importing the unmodified system Dag took 30.505 seconds and loaded google.cloud.aiplatform_v1beta1. This exceeds Airflow's default dagbag_import_timeout of 30 seconds and can surface as a Dag import timeout or a third-party circular-import failure.

Steps to reproduce:

  1. Check out main with the Google provider development dependencies installed.

  2. Run the Vertex AI Feature Store system Dag import test in Breeze:

    BACKEND=sqlite breeze run pytest airflow-core/tests/unit/always/test_example_dags.py \
        -k 'test_should_be_importable and vertex_ai_feature_store' -xvs
    
  3. Observe that parsing the Dag loads google.cloud.aiplatform_v1beta1 at module scope and may exceed the 30-second Dag import timeout.

What you think should happen instead?

Scanning the system Dag should not load the Vertex AI Feature Store client package. Heavy Google clients and protobuf-related imports should be deferred until task execution so the Dag remains importable within the default timeout.

Operating System

macOS host with the Apache Airflow Breeze CI container, Python 3.10.

Deployment

Other Docker-based deployment

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

After removing the parse-time aiplatform_v1beta1 imports, the system Dag imports without loading that root package and passes Airflow's generic system Dag import, database-query, and connection-access checks.

Are you willing to submit PR?
  • Yes I am willing to submit a PR!
Code of Conduct
Dominant language
Python
Stars
46.9k
Forks
17.9k
Avg merge
2d 5h
Merged PRs (30d)
480

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/airflow

All issues in apache/airflow

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.