apache/airflow

Set up multiple Airflow instances under the same domain with different sessions

Open

#17,689 opened on Aug 18, 2021

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Python (44,809 stars) (16,781 forks)batch import
area:webservergood first issuekind:documentationpriority:low

Description

Apache Airflow version: v2.1.1

OS: Ubuntu 20.04.2 LTS

Apache Airflow Provider versions: Not relevent

Deployment: AWS EC2 Instances

What happened: When using multiple Airflow instances (i.e. Production instance and a Development instance) the webserver navigation/redirection can send you back to the incorrect instance.

What you expected to happen: I expect when I press "Back" or "Save" within Airflow (for instance, when editing a 'pool' from the GUI' that it goes back to the previous page (of the instance I am working on, not a different instance that I happen to also be working with at the same time).

How to reproduce it: It seems both webserver instances are using the same cookie or session information. I am using a single NGINX reverse proxy to access both a dev instance and a prod instance (but with a unique URL to differentiate the two)..

Example URLs: https://ec2-myIP.eu-central-1.compute.amazonaws.com/airflow [ points to the PRODUCTION instance ] https://ec2-myIP.eu-central-1.compute.amazonaws.com/airflow_dev [ points to the DEVELOPMENT instance ]

Example ISSUE (using pools as an example, but it is not limited there):

  1. On PRODUCTION instance browse to Admin->Pools
  2. On DEVELOPMENT instance browse to Admin->Pools
  3. On PRODUCTION instance click Airflows Back navigation arrow/button
  4. On DEVELOPMENT instance browse to Admin->Pools, edit a pool, then click Airflows Back navigation arrow/button

Note that the "Save" button can redirect back to the other instance as well (just depends).

Anything else we need to know: Issue occurs frequently when using two independent instances of Airflow. There are no logs to speak of.

Are you willing to submit a PR?

Contributor guide

Tech stack
pythonflask
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Airflow webserver configurationHTTP session/cookie conceptsReverse proxy usage (NGINX)
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate Airflow's Flask session configuration, specifically the SESSION COOKIE PATH or SESSION COOKIE DOMAIN settings in the webserver. Check if the base url configuration (airflow.cfg) affects cookie path. Look at the webserver code in airflow/www/ for any hardcoded path references. Consider previous similar issues or PRs (e.g., #12345) that addressed session isolation. Reproduce locally with two Airflow instances behind NGINX with different URL prefixes to confirm the cookie collision and test potential fixes like setting separate cookie paths per instance.
Set up multiple Airflow instances under the same domain with different sessions · apache/airflow#17689 | Good First Issue