apache/airflow

Airflow REST API: Add generic capability to retrieve required information for the specified entity..

Open

#29,893 opened on Mar 3, 2023

View on GitHub
 (15 comments) (0 reactions) (0 assignees)Python (16,781 forks)batch import
area:APIgood first issuekind:feature

Repository metrics

Stars
 (44,809 stars)
PR merge metrics
 (Avg merge 7d 18h) (834 merged PRs in 30d)

Description

Description

Airflow REST API should add a generic capability to retrieve the required information only. Instead sending all of them. E.g. If I want to retrieve DAG Run details using REST API: https://airflow.apache.org/api/v1/dags/{dag_id}/dagRuns/{dag_run_id} Or want to fetch list of DAGs using RET API: https://airflow.apache.org/api/v1/dags

It always returns the full details. Often it is the case where caller is not interested in all the information.

So I suggest to add a generic capability to retrieve only needed information just like offset and limit. E.g. if we pass fields = {dag_id, is_paused} in the query parameter while calling https://airflow.apache.org/api/v1/dags API, So it returns JSON body contains {dag_id, is_paused} fields.

Similarly it is true for other end points as well (At least Get Ones)

Use case/motivation

  1. Optimize the information what we want to retrieve from server.
  2. Saving Network bandwidth by reducing the information to required one.
  3. Possibility to collect more data in one go.

Related issues

Cannot say.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Research direction
Investigate how to add a 'fields' query parameter to Airflow REST API endpoints by modifying the serializers to filter output based on requested fields.
Tech stack
python
Domain
backendapi
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonAirflowREST API
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60