Messures and Doc for index freshness/update · apache/pinot#14371

(2 Kommentare) (0 Reaktionen) (1 zugewiesene Person)Java (4.937 Stars) (1.234 Forks)batch import

documentationgood first issue

Beschreibung

A part of Pinots secret sauce for speed are indexes. Is there any info available what index freshness/delay is achievable for newly ingested data / just changed data (upsert) for the different kinds of indexes? on how this could be improved/reduced? On how index updating works for each type?

of course I found

some info snippets in some issues in this field:
- https://github.com/apache/pinot/issues/6431
- https://github.com/apache/pinot/issues/13504
some sentence on it in the doc for native text index https://docs.pinot.apache.org/basics/indexing/native-text-index
etc

but no overview / table summing this topic up.

Contributor Guide

Tech Stack: java
Domain: documentationdatabaseperformance
Issue Type: documentation
Schwierigkeit: 2
Geschätzte Zeit: 1-3 hours
Aktivitätsstatus: active
Klarheit: clear
Voraussetzungen: basic understanding of Pinot indexingaccess to documentation and issues
Einsteigerfreundlichkeit: 65
Research-Richtung: Review the existing documentation links and related issues (e.g., #6431, #13504) to understand current index freshness information. Then, study the code for each index type (e.g., text, inverted, star tree) to describe update mechanisms. Compile findings into a table or overview section in the documentation.