Bug: _check_skill_structure validates `body` (no frontmatter) instead of full skill text, causing false failures
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
Research direction
In evolution/skills/evolve_skill.py, inspect the baseline and evolved validate_all call sites around lines 122 and 189, then compare them with skill_module.py's raw/body definitions. Reproduce with --eval-source sessiondb or synthetic and confirm valid skills no longer produce evolved_FAILED.md and the constraint log passes skill_structure.
Written by the indexing model from the issue text.
Description
Summary
ConstraintValidator._check_skill_structure() always receives skill text without YAML frontmatter, so it perpetually fails with "Skill missing: YAML frontmatter (---)" even when the skill file is perfectly valid.
Root Cause
In evolution/skills/evolve_skill.py, two call sites pass the wrong slice of the parsed skill dict to validate_all():
Baseline check (line ~122):
# BUG: skill["body"] is the markdown body only — no frontmatter
baseline_constraints = validator.validate_all(skill["body"], "skill")
Evolved skill check (line ~189):
# BUG: evolved_body is also frontmatter-stripped; evolved_full has it
evolved_constraints = validator.validate_all(evolved_body, "skill", baseline_text=skill["body"])
skill_module.py documents the dict clearly:
"raw"— full file content (frontmatter + body)"frontmatter"— YAML between---markers"body"— markdown after frontmatter
_check_skill_structure looks for text.strip().startswith("---"), which can never match when called with "body".
Impact
- Every evolved skill is marked FAILED regardless of actual content quality
- Evolved output is saved as
evolved_FAILED.mdand never deployed - The constraint gate is entirely non-functional for the
skill_structurecheck baseline_textpassed to_check_growthis also wrong ("body"vs"raw"), making the growth-limit comparison slightly incorrect
Fix
# Line ~122 — baseline check
baseline_constraints = validator.validate_all(skill["raw"], "skill")
# Line ~189 — evolved check
evolved_constraints = validator.validate_all(evolved_full, "skill", baseline_text=skill["raw"])
evolved_full is already computed just above via reassemble_skill(skill["frontmatter"], evolved_body).
Steps to Reproduce
- Run any skill evolution with
--eval-source sessiondborsynthetic - Observe that evolved output always lands in
output/<skill>/evolved_FAILED.md - Check the constraint log —
skill_structurewill always show ✗
Environment
hermes-agent-self-evolutioncloned from main (tested 2026-05-10)- DSPy fallback to MIPROv2 (GEPA unavailable in installed version)
- Both
video-transcodingandaudio-transcriptionskills affected
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 638
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NousResearch/hermes-agent-self-evolution
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
NousResearch/hermes-agent-self-evolution#164 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
NousResearch/hermes-agent-self-evolution#93 · 4 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
All issues in NousResearch/hermes-agent-self-evolution
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100