facebook/duckling

Arabic regex match some parts of words that mustn't be matched

Open

#131 创建于 2017年12月24日

在 GitHub 查看
 (4 评论) (5 反应) (0 负责人)Haskell (4,282 star) (737 fork)batch import
bughelp wanted

描述

In some Arabic dimensions, the regex doesn't match whole words only, instead it sometimes match some part of the word that mustn't be matched at all ! which makes serious buggy outputs. I noticed this problem in the following dimensions: 1- ordinal 2- number 3- time 4- duration

for example: if the input text was: "وأحدث ذلك مشكلة كبيرة" which in English means: "And that made a big problem"

Duckling match the word "احد" (which means "Sunday") from its built-in regex, with the word "وأحدث" (which means "made") from the input text ! (see the following photo for run time) time bug

this was in "time" dimension.

examples for the other dimensions are also attached:

a- "ordinal" dimension: ordinal bug

b- "number" dimension: This example shows bugs in more the one dimension, number dim is one of them. number bug

贡献者指南

技术栈
无
领域
backend
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
needs investigation
前置要求
basic regexArabic language
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
30
研究方向
Investigate the Arabic regex patterns in Duckling's time, ordinal, number, and duration dimensions. Look at the relevant source files in the repository (likely under `duckling/`). The issue shows partial word matching; consider adding word boundaries. Check the maintainer comments for any suggested approach.
Arabic regex match some parts of words that mustn't be matched · facebook/duckling#131 | Good First Issue