add custom lexicon
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
Research direction
Start by reading test_regex.py and ontology_manager.py, especially the current pattern-matching path for lexicon or ontology terms. Trace how a detected term is classified, then define the custom dictionary behavior using the provided example and add coverage for exact matching. Done means custom entries are recognized with their configured labels without breaking existing matching.
Written by the indexing model from the issue text.
Description
Add a custom lexicon to test_regex.py or even better to ontology_manager.py. Currently ontology_manager.py using a pattern match to find words potentially in a lexicon/ontology. You might want to have an exact dictionary to do a lexcion match, like "Barack Obama": "PUBLIC_FIGURE"}
- Dominant language
- Python
- Stars
- 9
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/pii_processing
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 40/100
All issues in bigscience-workshop/pii_processing
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100