[bug-hunter] docs-edit-automation treats partial metadata updates as complete

Open Beginner friendly
#18,276 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
bash, github-actions
Domain
ci-cd

Research direction

Start with .github/workflows/docs-edit-automation.yml around lines 93-101 and compare its metadata classification with the reproduction script in the issue. Run the provided script and failing assertion for the partial-update case, then verify cases with neither and both metadata files changed. Done means only packages with both manifest.yml and changelog.yml changes are marked complete.

Written by the indexing model from the issue text.

Description

Impact

PRs that update package docs and only one metadata file (manifest.yml or changelog.yml) are marked as already complete by the docs helper workflow. That skips follow-up guidance even though one required metadata update is still missing, causing incomplete docs-update PRs and extra review churn.

Reproduction Steps

  1. Create and run this new minimal reproduction script:
cat > /tmp/gh-aw/agent/repro_docs_edit_automation.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""

while IFS= read -r doc_file; do
  [ -z "$doc_file" ] && continue
  PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
  MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
  CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)

  if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
    PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
  else
    PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
  fi
done <<< "$CHANGED_DOCS"

printf 'PACKAGES_NEEDING_UPDATE=%q\n' "$PACKAGES_NEEDING_UPDATE"
printf 'PACKAGES_ALREADY_DONE=%q\n' "$PACKAGES_ALREADY_DONE"
EOF
chmod +x /tmp/gh-aw/agent/repro_docs_edit_automation.sh
/tmp/gh-aw/agent/repro_docs_edit_automation.sh

Expected vs Actual

Expected: when docs changed and only one of manifest.yml / changelog.yml changed, the package should still be in PACKAGES_NEEDING_UPDATE.

Actual: package is incorrectly marked done:

PACKAGES_NEEDING_UPDATE=''
PACKAGES_ALREADY_DONE=\ sample

Failing Test

#!/usr/bin/env bash
set -euo pipefail

ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""

while IFS= read -r doc_file; do
  [ -z "$doc_file" ] && continue
  PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
  MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
  CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)

  if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
    PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
  else
    PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
  fi
done <<< "$CHANGED_DOCS"

# This assertion fails due to current workflow logic.
test -n "\$\{PACKAGES_NEEDING_UPDATE// }"

Evidence

  • Workflow logic classifies package as complete if either metadata file changed:
    • .github/workflows/docs-edit-automation.yml#L93-L101
  • Workflow summary text states completion means both were updated:
    • .github/workflows/docs-edit-automation.yml#L207 ("updated changelogs and manifests")

The classification condition should require both files to be present before adding a package to PACKAGES_ALREADY_DONE.

[!NOTE]

🔒 Integrity filtering filtered 31 items

Integrity filtering activated and filtered the following items during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

  • issue:elastic/integrations#unknown (search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18275 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18268 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18267 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18265 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18264 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18263 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18256 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18261 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18247 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18242 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18241 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18238 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18236 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18233 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/integrations#18230 (list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • ... and 15 more items

What is this? | From workflow: Bug Hunter

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  • expires on Apr 15, 2026, 11:33 AM UTC
Dominant language
Handlebars
Stars
334
Forks
652
Avg merge
1d 22h
Merged PRs (30d)
241

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from elastic/integrations

All issues in elastic/integrations

Similar issues

More DevOps issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.