add backtrans for regex pipeline

Open
#6 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
security

Research direction

Start in test_regex.py and read the existing regex pipeline to identify its translation and NER entry points. Trace the current test flow, then verify that it translates to English, performs NER, translates back to target_lang, and matches entities between the back-translated and original text.

Written by the indexing model from the issue text.

Description

in test_regex.py, add ability to do backtrans, first trans to english, perform ner, then translate back to target_lang, and match the ner entity between back transed text and original text.

Dominant language
Python
Stars
9
Forks
6
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bigscience-workshop/pii_processing

All issues in bigscience-workshop/pii_processing

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.