cannot use `get_current_context()` in a `@task.virtualenv` task · apache/airflow#34158

(14 评论) (1 反应) (0 负责人)Python (16,781 fork)batch import

area:providersgood first issuekind:featureprovider:standard

仓库指标

Star: (44,809 star)
PR 合并指标: (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

On Airflow 2.5.1 (on AWS MWAA), I ran this DAG:

from datetime import datetime, timedelta
from airflow.decorators import dag, task
from airflow.operators.python import get_current_context

@task.virtualenv(system_site_packages=True)
def test():
    data_interval_end = get_current_context()['data_interval_end']
    print(data_interval_end)

@dag(
    start_date=datetime(2023, 9, 6),
    schedule="10 * * * *",
)
def bug_test():
    test()

the_dag = bug_test()

And I got airflow.exceptions.AirflowException: Current context was requested but no context was found! Are you running within an airflow task?

I know that I can do it like this:

@task.virtualenv(system_site_packages=True)
def test(data_interval_end=None):
    print(data_interval_end)

That works fine if I only need the context directly inside that function, but where this actually popped up in practice was a DAG that used some shared lib functions that used get_current_context, which of course works fine when called from normal tasks but blew up when called from a virtualenv task.

What you think should happen instead

ideally, get_current_context() should work even if it's called from a virtualenv task.

How to reproduce

described above

Operating System

Linux? it's AWS MWAA

Versions of Apache Airflow Providers

No response

Deployment

Amazon (AWS) MWAA

Deployment details