GLiNER detection fails when response text contains HTML comments

Open
  • Is something wrong?
#291 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Start with the reproducer for TextResponseRecipe in data_designer.engine.models.recipes.response_recipes and trace the reported path through response_recipes.py and parser.py's lookup_parser. Run the synthetic GLiNER-style response with an HTML comment; done means detector responses preserve that entity text without the reported parsing failure.

Written by the indexing model from the issue text.

Description

Priority Level

Medium (Annoying but has workaround)

Describe the bug

During ATIF scrubbing, _raw_detected_entities fails with:

Non-retryable failure on _raw_detected_entities[rg=0, row=32]:
sequence item 3: expected str instance, _cython_3_2_9.cython_function_or_method found

We reproduced the exact exception locally with a synthetic GLiNER-style JSON response containing an HTML comment. The original failing response from the Spark run has not been captured, so that specific trigger remains unconfirmed.

Steps/Code to reproduce bug
from data_designer.engine.models.recipes.response_recipes import TextResponseRecipe

TextResponseRecipe().parse('{"entities": [{"text": "<!-- Alice -->"}]}')

Relevant traceback:

response_recipes.py:49, in <lambda>
    return lambda x: parser.parse(x).response
parser.py:221, in parse
    parsed_block = self.lookup_parser(element)(element)
parser.py:161, in lookup_parser
    tag_path = ".".join(lineage)
TypeError: sequence item 3: expected str instance, _cython_3_2_9.cython_function_or_method found
Expected behavior

Detector responses should preserve entity text containing HTML comments without failing.

Additional context

Locally reproduced with:

  • NeMo Anonymizer 0.4.0
  • Data Designer Engine 0.9.3
  • lxml 6.1.3

Anonymizer defines _raw_detected_entities as an LLMTextColumnConfig, which uses Data Designer's TextResponseRecipe. That recipe parses the response as Markdown/HTML even though it ultimately returns the original response. An lxml comment node has a function-valued .tag, causing the string join to fail.

Could the text recipe pass the raw response through without HTML parsing, or could the detector path bypass that parsing? This appears to originate in the Data Designer dependency, but surfaces during Anonymizer detection.

Dominant language
Python
Stars
133
Forks
22
Avg merge
1d 21h
Merged PRs (30d)
31

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA-NeMo/Anonymizer

All issues in NVIDIA-NeMo/Anonymizer

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.