Option to disallow (or penalise) typos on the first character of a token

Aperta
  • Qualcosa non va?
#3,078 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
55/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
backend-api-design, search

Direzione di ricerca

Start by reproducing the documented curl search against a collection using query_by, num_typos, and prefix:false, then compare first-character and later-character substitutions. Trace the search typo-tolerance entry point that processes these parameters. Done means an opt-in setting rejects or doubles the cost of first-character typos while preserving ordinary typo matching and the min-word-size behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

feature-request

Description

Please add an option to disallow (or penalise) typos on the first character of a token.

Today a substitution on the first character is treated like any other typo, so short words that differ only in their first letter match each other: shanel → chanel, boat → coat, mice → rice. In product search these are almost always different words, not misspellings. People rarely mistype the first letter of what they are searching for.

Algolia handles this by counting a typo on the first letter as two typos. With the usual minWordSizefor1Typo / minWordSizefor2Typos of 5 / 8, a first-letter typo is therefore rejected for any word shorter than 8 characters. We are migrating from Algolia, and this is one of the largest remaining relevance differences we see.

Steps to reproduce (v30.2)

curl -X POST "$TS/collections" -H "X-TYPESENSE-API-KEY: $KEY" -d '{
  "name": "repro", "fields": [{"name": "title", "type": "string"}]
}'
curl -X POST "$TS/collections/repro/documents/import?action=create" -H "X-TYPESENSE-API-KEY: $KEY" --data-binary '
{"id":"1","title":"chanel lipstick"}'
q (query_by: title, num_typos: 1, prefix: false) typo position Typesense Algolia (same settings)
chanal 5th character matches chanel matches (1 typo)
shanel 1st character matches chanel 0 hits

Measured on a real catalog: Algolia returned 0 for shanel, while Typesense returned all 1,097 chanel products.

Expected behaviour

An opt-in search parameter, for example either of these:

  • typo_first_char: false: never accept a typo on the first character;
  • first_char_typo_cost: 2: count a first-character typo as two (Algolia's behaviour), so min_len_2typo governs it.

Why it matters beyond single words

This interacts with synonyms and drop_tokens (see #552): once a synonym expansion has been reduced to one word, a first-letter typo of that word can pull in unrelated documents. In Thai, for example, เสื้อ ("shirt") matched เนื้อ ("texture") this way, and the two differ only in their first consonant. Rejecting first-character typos would stop that whole class of noise, while keeping typo tolerance for the misspellings people actually make.

Lingua principale
C++
Stelle
27.1k
Fork
1k
Merge medio
4g 4h
PR unite (30g)
15

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di typesense/typesense

Tutte le issue di typesense/typesense

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.