Messures and Doc for index freshness/update · apache/pinot#14371

(2 commentaires) (0 réactions) (1 assigné)Java (4 937 stars) (1 234 forks)batch import

documentationgood first issue

Description

A part of Pinots secret sauce for speed are indexes. Is there any info available what index freshness/delay is achievable for newly ingested data / just changed data (upsert) for the different kinds of indexes? on how this could be improved/reduced? On how index updating works for each type?

of course I found

some info snippets in some issues in this field:
- https://github.com/apache/pinot/issues/6431
- https://github.com/apache/pinot/issues/13504
some sentence on it in the doc for native text index https://docs.pinot.apache.org/basics/indexing/native-text-index
etc

but no overview / table summing this topic up.

Guide contributeur

Stack technique: java
Domaine: documentationdatabaseperformance
Type d'issue: documentation
Difficulté: 2
Temps estimé: 1-3 hours
Statut d'activité: active
Clarté: clear
Prérequis: basic understanding of Pinot indexingaccess to documentation and issues
Accessibilité débutant: 65
Direction de recherche: Review the existing documentation links and related issues (e.g., #6431, #13504) to understand current index freshness information. Then, study the code for each index type (e.g., text, inverted, star tree) to describe update mechanisms. Compile findings into a table or overview section in the documentation.