google-research/perch-hoplite
Ver no GitHubInvestigate better threading in SQLIteUsearchImpl
Open
#127 aberto em 12 de mai. de 2026
good first issue
Métricas do repositório
- Stars
- (115 stars)
- Métricas de merge de PR
- (Métricas PR pendentes)
Description
Currently, we make a thread_split copy of the DB for each thread. Phil points out that we might be able to use the thread-local cursor instead, and save a whole pile of overhead.
From Phil's modified SQLiteUsearchImpl:
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Create a thread-local storage container
self._thread_local = threading.local()
def _get_cursor(self) -> sqlite3.Cursor:
"""
Override: Returns a thread-local cursor.
If the current thread doesn't have one, it creates it.
"""
if not hasattr(self._thread_local, 'cursor'):
self._thread_local.cursor = self.db.cursor()
return self._thread_local.cursor