[Regression on closed #559] syncRepository still writes lastSyncedAt=now before syncIssues uses it as `since`, so every sync imports zero issues

Đang mở
  • Có gì không ổn?
#727 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
67/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
github, typescript
Lĩnh vực
api, backend, databases

Hướng nghiên cứu

Read src/github/github-sync.service.ts, focusing on syncRepository() and syncIssues(), then review src/github/github-sync.incremental.spec.ts to see the existing incremental sync coverage. Add an end-to-end service test that checks the since parameter uses the previous successful sync time, and is absent on the first sync. Done when the regression is covered and the sync imports issues on its first run.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

bug

Location

  • src/github/github-sync.service.ts:143-164 — GithubSyncService.syncRepository()
  • src/github/github-sync.service.ts:195-224 — GithubSyncService.syncIssues()

Problem

syncRepository stamps the repository row with lastSyncedAt: new Date() and persists it before calling syncIssues, and syncIssues then reads that very field to build GitHub's since filter.

In syncRepository:

const attrs = {
  githubRepoId: String(repoData.id),
  ...
  lastSyncedAt: new Date(),        // line 153
};

repository = repository
  ? this.repositoryRepo.merge(repository, attrs)
  : this.repositoryRepo.create(attrs);
repository = await this.repositoryRepo.save(repository);   // line 159
...
const result = await this.syncIssues(repository, owner, repo, page);  // line 164

In syncIssues:

const since =
  page === 1 && repository.lastSyncedAt instanceof Date
    ? repository.lastSyncedAt.toISOString()      // line 207 — this is "now"
    : undefined;

The in-memory repository object handed to syncIssues carries the timestamp that was created microseconds earlier, so since is always approximately the current instant. GET /repos/{owner}/{repo}/issues?since=<now> returns only issues updated after that instant — in practice an empty array.

This is the exact defect described by closed issue #559, and the closely related #560 (lastSyncedAt advancing before the sync's outcome is known). The incremental-sync behaviour added for #315 is intact; what regressed is the ordering in the caller.

Why it matters

POST /api/github/sync/:owner/:repo is the only path that imports issues into the platform. With this ordering:

  • A brand-new repository's first-ever sync creates the Repository row and imports zero Issue rows. There is nothing for a sponsor to post a bounty against.
  • Every subsequent re-sync also imports zero issues, because each call re-advances lastSyncedAt to the moment just before it queries.
  • The failure is completely silent and looks like success: the route returns { repository, synced: 0 } with HTTP 201, and the log line reads Synced 0 issues for owner/repo (page 1), which is indistinguishable from a genuinely up-to-date repository.

Webhook-driven issue updates (handleIssueEvent -> upsertIssueRecord) still work for issues that are already tracked, which masks the problem further: a repository can appear to be partially live while its backlog was never imported at all.

Reproduction / failure path

  1. Point the backend at an empty database and a GITHUB_API_TOKEN with read access to a public repo that has open issues (e.g. 20+).
  2. Authenticate as a MAINTAINER who passes RepoScopeGuard (add the repo to GITHUB_SYNC_ALLOWED_REPOS).
  3. POST /api/github/sync/<owner>/<repo>.
  4. Observed: HTTP 201 with synced: 0, a repositories row created with lastSyncedAt set, and SELECT count(*) FROM issues returning 0. The outbound Octokit request carries since=<the timestamp just written>.
  5. Expected: the repo's issues imported on the first call; on a later call, only issues changed since the previous successful sync.

Note that src/github/github-sync.incremental.spec.ts does not catch this: all of its cases (lines 93-130) call syncIssues directly with a hand-constructed Repository object and never go through syncRepository, so the ordering bug lives entirely in the untested seam between the two methods.

Suggested fix

Capture the previous watermark before overwriting it, and only advance the stored value after the page actually succeeds (which also addresses #560's concern):

const previousSyncedAt = repository?.lastSyncedAt ?? null;

const attrs = { /* ... no lastSyncedAt here ... */ };
repository = repository ? merge(...) : create(...);
repository = await this.repositoryRepo.save(repository);

const result = await this.syncIssues(
  { ...repository, lastSyncedAt: previousSyncedAt } as Repository,
  owner, repo, page,
);

// Only now, and only when the page completed without an interruption and
// there is no further page to walk, advance the watermark:
if (!result.nextPage) {
  await this.repositoryRepo.update({ id: repository.id }, { lastSyncedAt: syncStartedAt });
}

Using the timestamp taken before the first API call (syncStartedAt) rather than new Date() after the fact also avoids silently skipping issues that changed while the sync was running. Add a github-sync.service.spec.ts case that calls syncRepository end-to-end against a mocked Octokit and asserts the since parameter equals the stored lastSyncedAt from the previous run (and is absent on the very first run) — the ordering, not just syncIssues in isolation.

Ngôn ngữ chính
TypeScript
Star
2
Fork
90
Merge trung bình
31 phút
Pull request đã merge (30 ngày)
20

Chuẩn bị môi trường

  • Có Dockerfile hoặc tệp Docker Compose
  • Không có mẫu pull request
  • Không có hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của MergeFi/backend

Tất cả issue của MergeFi/backend

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.