Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

upsertPricePoints can synthesize an event id from the wall clock, breaking dedupe

Đã đóng
  • Có gì không ổn?
#620 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 3 ngày

@sparkysolutions10-pixel đang làm issue này rồi.

Từ ngày 25/10/2026.

  • #621 của @sparkysolutions10-pixel — đã merge

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
55/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
postgresql, typescript
Lĩnh vực
backend, database, testing

Hướng nghiên cứu

Start with src/db.ts and trace upsertPricePoints into synthesizeEventId; compare the timestamp fallback with the identity invariant in prisma/schema.prisma. Add a regression case in src/__tests__/backfill.test.ts for identical points without timestamp or observedAt, then run the full npm test suite. Done when the no-anchor behavior is explicit and repeated ingests preserve idempotency.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Background

prisma/schema.prisma states the invariant the price-point identity index depends on:

sources with none — AMM/Soroswap pool snapshots, Aquarius, anything added later — get a key synthesized deterministically from the row's content, never from a clock.

synthesizeEventId honours that by hashing the row's content, including the timestamp:

const content = [ network, p.source, p.pairKey, p.poolId ?? '', String(p.ledger),
                  p.timestamp.toISOString(), String(p.price), ... ]

#394 introduced a fallback in upsertPricePoints (src/db.ts):

const observedAt = p.observedAt ?? p.timestamp ?? new Date()
const timestamp  = p.timestamp ?? observedAt
...
eventId: p.eventId || synthesizeEventId(network, { ...p, timestamp })

When a point arrives with neither timestamp nor observedAt, timestamp becomes new Date() — so the synthesized event id now incorporates the wall clock. Two ingests of byte-identical data produce different event ids, the (network, source, eventId) identity index does not match, and skipDuplicates does not skip. The row is inserted twice.

That is the exact failure the identity work in #237 was built to make impossible, and it inflates volume and skews VWAP — silently, because nothing errors.

How it was found

Lens #385 (historical backfill) has a test that is the clearest statement of the contract:

proves idempotency on re-run: second run inserts 0 points and changes nothing
  AssertionError: expected 1 to be +0

It passed on that branch against main as of this morning and fails after merging current main. The branch is not at fault — its test is the thing that noticed.

Why it is subtle

Before #394, a point with no timestamp would have thrown on p.timestamp.toISOString(). #394 turned a loud crash into silent non-determinism, which is the worse of the two.

What to build
  • Decide what a point with no timestamp means. If every real ingester always sets one, the fallback should be an error rather than a clock read — a crash is recoverable, a duplicated money row is not.
  • If a clock fallback must stay, it must not reach synthesizeEventId. The id has to be derived only from fields that are stable across runs, and a row with no stable temporal anchor cannot be given a deterministic identity at all — it should be rejected rather than assigned a random one.
  • Either way, observedAt itself may keep the new Date() default; it is "when Lens saw it" and is not part of the identity.
Acceptance criteria
  • A point ingested twice with identical content produces the identical eventId, with no timestamp and no observedAt supplied
  • A test that fails against current main and passes after
  • The behaviour for a point with no temporal anchor is explicit — documented, and either rejected or given a non-clock identity
  • src/__tests__/backfill.test.ts's idempotency case passes in a full npm test run, not only in isolation
Notes

Found while reviewing #385, which is blocked on it. That PR should not be asked to fix this — it is a regression in main.

scripts/migrations/2026-10-17-price-point-identity.sql and the @@unique in prisma/schema.prisma are the surrounding context.

Ngôn ngữ chính
TypeScript
Star
4
Fork
71
Merge trung bình
3 ngày 10 giờ
Pull request đã merge (30 ngày)
154

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của Divinity-Labs/Lens

Tất cả issue của Divinity-Labs/Lens

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.