Execution.create() fails with 503 due to credentials annotation typo

Open Beginner friendly
#6,610 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
google-cloud, python

Research direction

Start in google/cloud/aiplatform/metadata/execution.py at the create() and _create() definitions, then compare their credentials parameters and initialization flow with artifact.py and context.py. Run the supplied standalone Execution.create() reproduction; done means the annotation and default metadata-store setup match the issue’s requested behavior and the call no longer fails with the reported 503.

Written by the indexing model from the issue text.

Description

api: vertex-ai

Bug

google.cloud.aiplatform.metadata.execution.Execution.create() fails with a 503 error when called without explicitly passing credentials=:

503 Getting metadata from plugin failed with error: before_request
Root cause

Two issues in google/cloud/aiplatform/metadata/execution.py:

1. credentials parameter annotation typo (high severity)

Both create() and _create() use = instead of : for the credentials type annotation:

# Current (broken) -- execution.py lines 103, 181:
credentials=Optional[auth_credentials.Credentials],

# Should be (matches artifact.py, context.py):
credentials: Optional[auth_credentials.Credentials] = None,

Because = is used instead of :, the default value is the typing.Optional[google.auth.credentials.Credentials] type object itself (a _UnionGenericAlias), not None. This non-None type object is passed down to the gRPC auth stack, which attempts to call .before_request() on it, producing the 503.

Introduced in PR #1410 (June 2022). artifact.py and context.py both have the correct : ... = None syntax.

2. Missing ensure_default_metadata_store_exists() call (lower severity)

Artifact.create() calls metadata_store._MetadataStore.ensure_default_metadata_store_exists() before delegating to _create(). Execution.create() skips this call entirely. This means standalone Execution.create() will also fail if the default metadata store hasn't been implicitly created by a prior Artifact.create() or aiplatform.init(experiment=...) call. Context.create() has the same gap but is not addressed here.

Reproduction
from google.cloud import aiplatform
from google.cloud.aiplatform.metadata import execution as metadata_execution
from google.cloud.aiplatform.compat.types import execution as gca_execution

aiplatform.init(project="my-project", location="us-central1")

# This fails with 503:
exec_obj = metadata_execution.Execution.create(
    schema_title="system.CustomJob",
    resource_id="test-execution",
    display_name="test",
    state=gca_execution.Execution.State.COMPLETE,
    metadata={"component_type": "custom_job"},
)
Workaround

Call ensure_default_metadata_store_exists() manually, which also initializes credentials properly:

from google.cloud.aiplatform.metadata import metadata_store
metadata_store._MetadataStore.ensure_default_metadata_store_exists(
    project="my-project", location="us-central1",
)
# Now Execution.create() works
Fix

Two changes to execution.py:

  1. Fix credentials=Optional[auth_credentials.Credentials] to credentials: Optional[auth_credentials.Credentials] = None on both create() and _create()
  2. Add the ensure_default_metadata_store_exists() call in create() (matching Artifact.create())
Environment
  • google-cloud-aiplatform 1.140.0
  • Python 3.13.2
  • macOS / Vertex AI training containers (both affected)
Dominant language
Python
Stars
907
Forks
467
Avg merge
1d 13h
Merged PRs (30d)
44

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from googleapis/python-aiplatform

All issues in googleapis/python-aiplatform

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.