Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

memory: keep a raw input log of /add requests, independent of extraction

Abierto
  • ¿Algo no está bien?
#537 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
api, backend

Línea de trabajo

Start in the POST /api/v2/memory/add route handler, where the request is validated and split into sync and async paths; the log write goes before that split. Read service/_boundary.py to see how the cell is built from CanonicalMessage.text, so you know what the raw log must capture instead. Done means one JSON line per accepted request under .input_log/YYYY-MM-DD.jsonl, with retention sweeping and base64 truncation working and log-write failures not failing /add. Coordinate with #483 before touching session deletion.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Problem

EverOS keeps no record of what a client actually sent to POST /api/v2/memory/add. The closest things are by-products of extraction:

  • unprocessed_buffer holds messages only until boundary detection folds them into a memcell.
  • memcell.payload_json keeps the conversation text, but not as it was sent. Attachments have already been replaced by their parsed text (service/_boundary.py builds the cell from CanonicalMessage.text), and the messages are grouped the way boundary detection cut them. It also lives in .index/sqlite/system.db, inside the directory that docs/how-memory-works.md says can be deleted without losing anything.
  • In async mode, memorize_inbox.payload_json is cleared as soon as the row is delivered.

So once extraction has run, the original input can no longer be inspected, replayed or audited. (EverOS Cloud reports every request body to a request log outside the engine.)

Proposal

An input log that has nothing to do with extraction:

  • What: every /add request that passes validation, as the client sent it (model_dump(mode="json", exclude_unset=True)), plus its request_id and receive time. One JSON line per request. /flush and /search are not logged.
  • When: in the route, before the sync / async split. Each accepted request is logged exactly once, whatever happens to extraction afterwards; queue retries do not log it again.
  • Where: <root>/<app>/<project>/.input_log/YYYY-MM-DD.jsonl. The date is in the display timezone, the same as the daily-log Markdown files. The directory is outside .index/, so deleting the index does not lose it. No memory kind matches the path, so the cascade scanner and watcher ignore it.
  • Retention: retention_days, default 30. A file whose date is older than that is deleted. The sweep runs when a scope writes its first line of a day, including the first line after a restart.
  • Large attachments: if a line would exceed max_inline_bytes (default 1 MiB), the base64 of each content item is replaced by its length and SHA-256. Text is always kept verbatim.
  • Failure: a failed log write is logged at error level and does not fail /add.
  • Config: [memorize.input_log] with enabled = true, retention_days = 30 and max_inline_bytes = 1048576.

Relation to #483

A session-only delete must also remove that session's lines from the input log. Otherwise "forget this conversation" leaves the raw text on disk for up to retention_days. That work belongs to #483.

Out of scope

  • Reading the log through the API.
  • Moving memcell / unprocessed_buffer out of .index/. The docs' claim that SQLite is rebuildable from Markdown already does not hold for them.
Lenguaje dominante
Python
Estrellas
13.3k
Forks
926
Merge medio
22 h 56 min
PR fusionados (30 d)
27

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de EverMind-AI/EverOS

Todos los issues de EverMind-AI/EverOS

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.