Is there any way to use "doc.spans" in 01_parse.py?
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Read 01_parse.py and the spaCy v3.1 changes referenced in the issue, focusing on doc.spans and the component's assignment to doc.ents. Determine whether overlapping spans can be handled by the existing parsing flow; done means providing a confirmed supported approach or documenting that this workflow cannot use overlapping spans.
Written by the indexing model from the issue text.
Description
Hi,
I am trying to built a sense2vec model with new data. I have made few changes in 01_parse.py.
First, I have removed the default ner pipe coming with "en_core_web_lg".
Then I have added a new Language.component where I identify Spans associated to a new entities (new labels) in a doc.
Sometimes, I would like to assign a Span[x, y] to more than one entity but I can not.
My question...
I have read the new changes in spaCy v3.1. Is there a way to use "doc.spans" (or something similar) in 01_parse where SpaCy's internal algorithms take Spans overlap into account?
@Language.component("name_comp")
def my_component(doc):
matches = matcher(doc)
seen_tokens = set()
new_entities = []
entities = doc.ents
for match_id, start, end in matches:
# check for end - 1 here because boundaries are inclusive
if start not in seen_tokens and end - 1 not in seen_tokens:
new_entities.append(Span(doc, start, end, label=match_id))
entities = [
e for e in entities if not (e.start < end and e.end > start)
]
seen_tokens.update(range(start, end))
doc.ents = tuple(entities) + tuple(new_entities)
return doc
Thanks in advance,
Paula
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 236
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from explosion/sense2vec
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
import error Openusage
Difficulty 4/5 3-5 days Newbie friendliness 20/100
-
usage
Difficulty 3/5 1-2 days Newbie friendliness 30/100
-
provide citation Open
Difficulty 1/5 Under an hour Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
All issues in explosion/sense2vec
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100