apache/airflow

unclear docs on custom decorators

Open

#33,167 创建于 2023年8月7日

在 GitHub 查看
 (8 评论) (0 反应) (0 负责人)Python (16,781 fork)batch import
good first issuekind:documentation

仓库指标

Star
 (44,809 star)
PR 合并指标
 (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

What do you see as an issue?

Referring to: https://airflow.apache.org/docs/apache-airflow/2.3.4/howto/create-custom-decorator.html

The documentation is unclear. It assumes that the reader has done a bunch of things that haven't been well defined and then the last section talks about the ProviderManager class but it's very brief. You're expected to already know what the ProviderManager is. If I use the search box in the docs then apparently ProviderManager is only mentioned in the create-custom-decorator page.

It seems like the only way to make use of this feature is to go digging through the source code. The operator decorator code is not the most straight-forward to read. I assume that most people wont be able to make use of this feature at all.

Solving the problem

Either turn it into a tutorial and walk the reader through the whole process in another way.

Anything else

This is a workaround that seems to do the trick:

I initially came across this because I wanted to make a django operator that would work with the taskflow api. I did this and it seems to work.

def make_django_decorator(app_path, settings_module):
    def django_task(task_id: str = None, *args, **kwargs):
        def django_task_decorator(fn):
            @task(task_id=task_id or fn.__name__, *args, **kwargs)
            def new_fn(*args, **kwargs):
                django_connect(app_path, settings_module)  # connect to the database. Now fn can make use of Django models
                return fn(
                    *args,
                )

            return new_fn

        return django_task_decorator

    return django_task


# I have a Django project named Tilde, now I have a tilde-specific task decorator. I should be able to make operatrs for other Django projects f I want to

tilde_task = make_django_decorator(
    app_path=TILDE_REPO_PATH / "backend", settings_module="backend.settings"
)

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

贡献者指南