ansible/awx

adhoc jobs block other jobs from being processed in the queue

Open

#14.645 geöffnet am 9. Nov. 2023

Auf GitHub ansehen
 (5 Kommentare) (1 Reaktion) (0 zugewiesene Personen)Python (13.071 Stars) (3.333 Forks)batch import
communitygood first issuehelp wantedtype:bug

Beschreibung

Please confirm the following

  • I agree to follow this project's code of conduct.
  • I have checked the current issues for duplicates.
  • I understand that AWX is open source software provided for free and that I might not receive a timely response.
  • I am NOT reporting a (potential) security vulnerability. (These should be emailed to security@ansible.com instead.)

Bug Summary

When you submit an adhoc job all other jobs (other adhoc, and job templates) are stuck in a pending state until the running adhoc job completes.

AWX version

23.4.0

Select the relevant components

  • UI
  • UI (tech preview)
  • API
  • Docs
  • Collection
  • CLI
  • Other

Installation method

kubernetes

Modifications

no

Ansible version

No response

Operating system

No response

Web browser

Firefox

Steps to reproduce

  1. submit any adhoc job (I used a shell command sleep 30; whoami)
  2. submit a job from "Templates"
  3. observe the job queue

Expected results

all jobs in the unified job queue should process according to capacity. I expect to see more than one job in a running state.

Actual results

The adhoc job is in a running state. All other jobs are stuck in pending waiting for the adhoc command to finish.

Additional information

I am using a container group to run jobs. Capacity of the managed nodes have been visually verified to be < 25% cpu and mem usage. I tried creating a duplicate container group for job templates, which still was stuck in pending, but now with a message waiting for adhoccommand-371 to finish

Contributor Guide

Tech Stack
pythondockerkubernetes
Domain
backenddevops
Issue Type
bug
SchwierigkeitGeschätzte Implementierungsschwierigkeit für neue Contributors, von 1 für sehr kleine Änderungen bis 5 für Expertenarbeit.
4
Geschätzte ZeitEin grober Zeitrahmen, um zu recherchieren, zu implementieren, zu testen und einen Pull Request vorzubereiten.
1-2 days
AktivitätsstatusWie verfügbar das Issue gerade wirkt: frisch, aktiv, stale, blockiert oder wartet auf Maintainer-Input.
stale
KlarheitWie klar das Issue die erwartete Änderung, Akzeptanzkriterien und den nächsten Schritt erklärt.
clear
Voraussetzungen
PythonAWX internalsKubernetes basicsConcurrency concepts
EinsteigerfreundlichkeitEin Score von 1 bis 100, der schätzt, wie zugänglich dieses Issue für First-Time Contributors ist.
25
Research-Richtung
Investigate the job queue scheduling logic in AWX, specifically how adhoc jobs are assigned to container groups. Look at the task manager code (likely in awx/main/dispatch/ or awx/main/tasks/) and the logic that transitions jobs from pending to running. The issue notes that all other jobs are stuck pending until the adhoc job finishes, suggesting a locking or capacity issue. Review the comments on the issue (#14645) for any additional insights and check the relevant test files for similar scenarios.