Two incompatible content_hash formulas cause unbounded re-import + re-embed

Version: 0.42.66.1 (Postgres engine, self-hosted)

Summary

content_hash is computed by two different formulas in the codebase. Pages written through one path are never recognised as unchanged by the other, so they are re-chunked (and, with embeddings on, re-embedded) on every import — forever.

location	formula
src/core/import-file.ts:572	{ title, type, compiled_truth, timeline, frontmatter, tags }
src/core/utils.ts:59 (contentHash)	{ title, type, compiled_truth, timeline, frontmatter } — no tags

postgres-engine.ts:1072 picks between them: const hash = page.content_hash || contentHash(page). So any caller that does not pass an explicit hash — gbrain put, and anything built on it — stores a hash the importer can never reproduce.

Why it matters

This is not theoretical. On a 13.8k-page brain it produced a loop that re-embedded 185,000 chunks/day against a baseline of 200–500. The pages affected were ones that had been touched by gbrain put (a bulk content-cleanup script) and by fact-absorption, i.e. exactly the pages the tooling writes to most often.

Each import saw a hash mismatch → rewrote the page → stored a hash from the other formula → next import saw a mismatch again. generation on the affected rows had climbed past 200.

Reproduce

1. Import a markdown file so the page exists with an importer-computed hash.
2. Write the same page via gbrain put (no explicit content_hash), so the engine falls back to utils.contentHash.
3. Re-run the import — the file on disk is byte-identical to compiled_truth in the DB, yet the page is re-imported.
4. Repeat: it never converges.

Verified by recomputing both formulas by hand against the stored value: the DB hash matches the no-tags variant, while the importer computes the tags-inclusive one.

Suggested fix

Have both paths call one shared helper. Since the importer's version is the stricter superset, extending utils.contentHash to include tags (sorted) looks like the smaller change — but it invalidates existing hashes, so it should land with a one-time rehash/migration, or the comparison should tolerate both during a transition window.

Related: two files silently resolving to one slug

Separately (and with the same symptom — endless re-import), two files in the same directory can resolve to the same slug with no warning:

private/tim/tasks/ценообразование-определить-себестоимость-и-цену-пр-25ccfd76c427.md
private/tim/tasks/25ccfd76c427.md

The long name is reduced to its hash suffix, colliding with the short file. Each import writes both, so each one's hash is always the "existing" value for the other:

file A: computed=919a49e7… existing=bc694698… match=false
file B: computed=bc694698… existing=919a49e7… match=false

It never converges, and nothing in the output hints that two files are fighting over one page — the run just reports N imported every time. On our brain this affected 23 pages.

A warning when two collected files map to the same slug in one import run would have made this a five-minute diagnosis instead of a long one. There is already a precedent for this kind of warning at import-file.ts:652 (shares content_hash with …).

Secondary observation

content_chunks.model is stamped at chunk creation and never updated on re-embed, and its column default is stale (qwen3-embedding:0.6b). After migrating embedding providers, ~half our chunks still carried the previous provider's label while holding vectors from the current model — verified by cosine similarity of 0.9999 between identically-texted chunks bearing different labels. Harmless at query time, but it makes model unusable for auditing which chunks actually need re-embedding.