Multilingual typing: committed words are learned in the user history of the wrong language

未关闭
  • 哪里不对？
#763 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
62/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
android, kotlin
领域
internationalization, mobile

调研方向

Start in DictionaryFacilitatorImpl, where addToUserHistory() reads currentlyPreferredDictionaryGroup before adjustConfidences() and then calls addWordToUserHistory with that group. Check unlearnFromUserHistory() as well, since it only touches the preferred group's history. Done when a German word typed after English words is stored in the de_DE history, and unlearnFromUserHistory() clears the word from every group that holds it. The reporter's groupForLearning idea is a starting point to verify, not a finished patch.

由索引模型根据 Issue 内容生成。

描述

Describe the bug
With multilingual typing, DictionaryFacilitatorImpl.addToUserHistory() always adds the word to the user history of currentlyPreferredDictionaryGroup. It even reads that group before adjustConfidences() runs for the word:

val preferredGroup = currentlyPreferredDictionaryGroup   // read here
...
adjustConfidences(suggestion, wasAutoCapitalized)        // confidence changes afterwards
...
// add to history for preferred dictionary group, to avoid mixing languages in history
addWordToUserHistory(preferredGroup, ngramContextForCurrentWord, currentWord, ...)

So after a few English words, a German word, typed or picked from the suggestions, goes into the English user history. There:

  • the English main dictionary doesn't know it (mainFreq = -1), so it is stored as an invalid word and needs several uses before it's suggested;
  • it is scored with the English locale weight;
  • the bigram (previous word → German word) is stored in the English history too.

This is the opposite of what the comment intends ("to avoid mixing languages in history"). It also means that correcting a wrongly swiped word (e.g. German "hier" that came out as English "hitter") barely helps next time.

unlearnFromUserHistory() has the same issue: it only removes the word from the preferred group's history.

Log from our build (German main, English (US) secondary, with a temporary debug line in addToUserHistory). After "the weather is good and I think that", the user types "hier":

learn 'that' -> en_US, preferred en_US, confidences [de_DE=0, en_US=8]
learn 'hier' -> de_DE, preferred en_US, confidences [de_DE=1, en_US=2]

With the current LeanType code, "hier" goes to en_US, because that group is preferred. With our fix it goes to de_DE.

Fix that works for us
Pick the history by the word's language: the preferred group if the word belongs to it (or to no language), otherwise the group whose main/apps/personal dictionary contains it:

private fun groupForLearning(word: String): DictionaryGroup {
    val preferred = currentlyPreferredDictionaryGroup
    if (dictionaryGroups.size == 1) return preferred
    fun belongsTo(group: DictionaryGroup) = isValidWord(word, DICTIONARY_TYPES_EXCLUDING_HISTORY, group)
        || isValidWord(word.lowercase(group.locale), DICTIONARY_TYPES_EXCLUDING_HISTORY, group)
    if (belongsTo(preferred)) return preferred
    return dictionaryGroups.filter { it !== preferred && belongsTo(it) }.maxByOrNull { it.confidence } ?: preferred
}

Use it in addToUserHistory() (per word, after adjustConfidences()), and let unlearnFromUserHistory() remove the word from the history of all groups. I can send a PR once we've tested it a bit longer.

To Reproduce

  1. Languages & Layouts → German → Multilingual typing → add English (US). Main dictionaries for both, personalized suggestions on.
  2. Type a few English words, then a German word that is not English (e.g. "hier").
  3. The word is added to the English user history (visible with a debug log, or via the dictionary dump in the debug settings).

Expected behavior
Words are learned in the user history of their own language.

App version & Flavor
Found in GlassType, my fork, now based on LeanType 4.3.1 (0e673e1). The code is unchanged in current main.

Device Info

  • Model: Samsung Galaxy Z Fold 3 (SM-F926B)
  • OS version: Android 15
主要语言
Kotlin
星标
1k
派生
59
平均合并
14 小时 9 分钟
30 天内合并 PR
56

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

LeanBitLab/LeanType 的其他 Issue

查看 LeanBitLab/LeanType 的全部 Issue

相似的 Issue

更多 Kotlin Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。