apache/airflow

DagBag Processing SIGSEGV causes runaway tasks with LocalExecutor

Open

#57174 opened on Oct 23, 2025

View on GitHub
 (8 comments) (0 reactions) (0 assignees)Python (44,809 stars) (16,781 forks)batch import
area:Executors-corearea:coregood first issuekind:bug

Description

Apache Airflow version

3.1.0

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

Using Airflow 3.1.0 - airflow standalone with the LocalExecutor on MacOS 14 - M3 ARM

I've noticed that occasionally tasks that are dynamically created (with .expand(), I don't think this is the issue fyi, just including), a subset of the them can runway, and will not fail even with a execution_timeout set. It completely disregards the timeout after this error is found in the logs:

ERROR - Received SIGSEGV signal while processing <my dag path>. source=airflow.models.dagbag.DagBag loc=dagbag.py:375

Here's some screenshots from a manually failed run, note the duration of 30 minutes even though a max timeout of 3 minutes:

Log from the Task:

Dag definition (linting errors are unrelated to this issue and are in a different task.):

I have a hunch this happens when the DAG Processor starts running at the same time as all of the tasks are dynamically created, but I truthfully have no clue.

I don't really have an issue with tasks like this failing on standalone due to a perfect storm of conditions (and non-intended OSes) as I only use it for local testing, but I would like the tasks to not run away.

Possibly Related Ticket: https://github.com/apache/airflow/issues/47011 https://github.com/apache/airflow/issues/55838 https://github.com/apache/airflow/issues/50303 https://github.com/apache/airflow/discussions/35967

What you think should happen instead?

No response

How to reproduce

No clue, besides try casting a wide net with many dynamically created tasks in the chance that this error bubbles up. I've included my DAG code besides my custom BoxHook in case this helps.

Operating System

MacOS 14 - M3 ARM

Versions of Apache Airflow Providers

apache-airflow-providers-google==18.0.0 apache-airflow-providers-standard==1.8.0

Deployment

Other

Deployment details

Local Standalone

Anything else?

I think it bubbles up in the DAG run about 20% of the time.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Tech stack
python
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
PythonAirflow DagBagSignal handlingConcurrency
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
10
Research direction
Investigate the SIGSEGV signal occurring in `airflow/models/dagbag.py` at line 375 during DagBag processing. Focus on the interaction between DAG parsing and task execution with LocalExecutor, particularly when tasks are dynamically created with `.expand()`. Examine the linked issues for similar patterns, and consider concurrency and signal handling in the DagBag code. The bug is intermittent and occurs on MacOS ARM, so environment specifics may be relevant.