Add the ResponseCache protocol, make_cache_key and InMemoryCache

Abierto
  • ¿Algo no está bien?
#145 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
56/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend, testing

Línea de trabajo

Start with lettucedetect/detectors/cache.py and the key construction sites in llm.py named in the issue; run uv run --extra dev pytest tests/test_llm_detector_pytest.py -q first. Implement and export the protocol, key helper, sync base, and in-memory backend in the named module, then add the reusable contract in tests/cache_contract.py and configure pytest in pyproject.toml. Done means the contract and existing suite pass, including key compatibility, TTL, async behavior, thread safety, and cancellation checks.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

ready-for-agent

Parent: #144 · Epic: #126 · Blocked by: #135, #131 (for the recorded key strings)
Branch from stateless_core; open the PR against it.

Problem

Caching is welded into LLMDetector through a concrete CacheManager, with keys built inline and no model version in the key, so a redeployed model serves stale replies. There is no protocol a service can implement.

Current behavior

  • lettucedetect/detectors/cache.py:28 _hash joins parts with || and hashes with SHA-256.
  • Keys are built inline: judge llm.py:408, native :468 (extra "native"), verify :506 (extra "verify"). No model version.
  • CacheManager.get/set/delete are sync only and have no TTL (cache.py:31-47).

Steps

  1. Add lettucedetect/detectors/response_cache.py with the ResponseCache protocol: get(key), set(key, value, *, ttl_seconds=None), delete(key) and one async def pair for each: aget(key), aset(key, value, *, ttl_seconds=None), adelete(key). Each async method takes the same arguments as its sync pair and returns the same value. Keys are opaque strings, values are strings, a missing or expired key returns None.
  2. Add make_cache_key(prompt, model, model_version, temperature, kind), kind in "judge" | "native" | "verify". With model_version=None it must equal today's _hash output (use the keys recorded in #131); with a version, add the version as an extra part.
  3. Add SyncCacheBase, a small base class for backends that only have sync storage: the backend implements get, set and delete, and the base supplies aget, aset (with ttl_seconds) and adelete by running them in a worker thread (asyncio.to_thread). InMemoryCache and FileCache (#146) build on it, so the async pairs are written once.
  4. Add InMemoryCache with TTL, thread-safe, on SyncCacheBase; aset honours ttl_seconds exactly like set.
  5. Add tests/cache_contract.py (value reads back, missing is None, TTL expiry, delete, each sync method equals its async pair, aset with ttl_seconds expires like set, a value written by set reads via aget and the reverse, after an aset task is cancelled, the cache is valid: the key reads as the old value or the new one, never a partial or corrupt value, and later get/set work; concurrent writers lose no key from threads and from asyncio.gather over aset) and run it against InMemoryCache.
  6. Add asyncio_mode = "auto" under [tool.pytest.ini_options] in pyproject.toml (pytest-asyncio is already in the dev extra) so the async tests are plain async def. If #141 already added the line, leave it.

Acceptance criteria

  • SyncCacheBase exists: a test backend that implements only get, set and delete on it passes the contract suite, async methods included, and a class implementing the protocol directly is not forced to use it.
  • Protocol, key function, SyncCacheBase and InMemoryCache exist and are exported from lettucedetect.detectors.
  • make_cache_key(..., None, ...) equals the recorded keys for all three kinds; different versions give different keys.
  • ResponseCache declares aget, aset and adelete as coroutine functions with the same arguments as get, set and delete; a backend lacking one fails a protocol check in the suite.
  • The async methods of InMemoryCache run off the event loop's thread, and the loop keeps running while they wait (ticker test).
  • The contract suite passes for InMemoryCache and is reusable for other backends (#146 uses it).
  • pyproject.toml sets asyncio_mode = "auto"; the async contract tests are plain async def, and the full existing suite still passes.
  • No detector changes; existing tests pass; follows CONTRIBUTING "Errors and logging".

Start here

uv run --extra dev pytest tests/test_llm_detector_pytest.py -q
Lenguaje dominante
Python
Estrellas
615
Forks
60
Merge medio
1 d 8 h
PR fusionados (30 d)
10

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de KRLabsOrg/LettuceDetect

Todos los issues de KRLabsOrg/LettuceDetect

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.