MarkdownHeaderSplitter treats headings inside longer closing fences as headers
Maintainers usually reply within 1 day
@julian-risch is already working on this.
Since Sep 27, 2026.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- backend, testing-qa
Research direction
Start at the MarkdownHeaderSplitter entry point and its _code_block_pattern, then reproduce the issue with the supplied Python example. Update the fence matching so longer backtick and tilde closers are excluded from header matches, and add regression coverage for longer and equal-length closers plus the real header after the block.
Written by the indexing model from the issue text.
Description
Problem
MarkdownHeaderSplitter excludes headers inside fenced code blocks only if the closing fence has exactly the same length as the opening fence. Markdown permits a closing fence of the same type with at least as many fence characters. With a longer valid closing fence, the splitter treats code comments as document headers and cuts the code block apart.
Reproduction
On current main:
from haystack import Document
from haystack.components.preprocessors import MarkdownHeaderSplitter
text = "# Intro\nBody\n````python\n# fake\n`````\n## Real\nEnd\n"
result = MarkdownHeaderSplitter().run(documents=[Document(content=text)])
print([doc.meta["header"] for doc in result["documents"]])
Actual: ['Intro', 'fake', 'Real']. Expected: ['Intro', 'Real']. I ran it twice with identical results. Replacing the five-backtick closer with a four-backtick closer gives the expected result. The same failure occurs with four opening tildes and five closing tildes.
The current _code_block_pattern uses a backreference to the opening fence, requiring equal lengths. The CommonMark rule is a closing fence of the same type with at least the opening length: https://spec.commonmark.org/0.31.2/#fenced-code-blocks .
Suggested scope
Recognize longer closing fences when excluding code blocks from header matches, while keeping unrelated heading and secondary-split behavior unchanged. Regression tests should cover longer backtick and tilde closers, equal-length closers, and the real header after the block. This should not require a public API change. I did not find a matching open/closed issue or PR and can prepare a patch once the current Haystack PR queue clears.
- Dominant language
- Python
- Stars
- 26.6k
- Forks
- 3.2k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 263
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepset-ai/haystack
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
deepset-ai/haystack#13029 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
deepset-ai/haystack#13022 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepset-ai/haystack#12994 · 1 assignee ·
Maintainers usually reply within 1 day
-
P3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepset-ai/haystack#12945 ·
Maintainers usually reply within 1 day
-
Sockets.__getattribute__ references _sockets but the actual attribute is _sockets_dict — the optimization at lines 127-132 is dead code due to a typoPossibly taken @julian-risch claimed this 4 days ago. OpenP2
Difficulty 1/5 Under an hour Newbie friendliness 91/100
deepset-ai/haystack#12939 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
All issues in deepset-ai/haystack
Similar issues
-
customer-reported
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Azure/azure-cli#34150 · 1 comment ·
Maintainers usually reply within 1 day
-
community-request
Difficulty 1/5 Under an hour Newbie friendliness 95/100
NVIDIA-NeMo/Curator#2464 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
WeblateOrg/translation-finder#1099 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
trezor/trezor-firmware#7997 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day