[Regression on closed #559] syncRepository still writes lastSyncedAt=now before syncIssues uses it as `since`, so every sync imports zero issues

Open
  • Is something wrong?
#727 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
67/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github, typescript
Domain
api, backend, databases

Research direction

Read src/github/github-sync.service.ts, focusing on syncRepository() and syncIssues(), then review src/github/github-sync.incremental.spec.ts to see the existing incremental sync coverage. Add an end-to-end service test that checks the since parameter uses the previous successful sync time, and is absent on the first sync. Done when the regression is covered and the sync imports issues on its first run.

Written by the indexing model from the issue text.

Description

bug

Location

  • src/github/github-sync.service.ts:143-164 — GithubSyncService.syncRepository()
  • src/github/github-sync.service.ts:195-224 — GithubSyncService.syncIssues()

Problem

syncRepository stamps the repository row with lastSyncedAt: new Date() and persists it before calling syncIssues, and syncIssues then reads that very field to build GitHub's since filter.

In syncRepository:

const attrs = {
  githubRepoId: String(repoData.id),
  ...
  lastSyncedAt: new Date(),        // line 153
};

repository = repository
  ? this.repositoryRepo.merge(repository, attrs)
  : this.repositoryRepo.create(attrs);
repository = await this.repositoryRepo.save(repository);   // line 159
...
const result = await this.syncIssues(repository, owner, repo, page);  // line 164

In syncIssues:

const since =
  page === 1 && repository.lastSyncedAt instanceof Date
    ? repository.lastSyncedAt.toISOString()      // line 207 — this is "now"
    : undefined;

The in-memory repository object handed to syncIssues carries the timestamp that was created microseconds earlier, so since is always approximately the current instant. GET /repos/{owner}/{repo}/issues?since=<now> returns only issues updated after that instant — in practice an empty array.

This is the exact defect described by closed issue #559, and the closely related #560 (lastSyncedAt advancing before the sync's outcome is known). The incremental-sync behaviour added for #315 is intact; what regressed is the ordering in the caller.

Why it matters

POST /api/github/sync/:owner/:repo is the only path that imports issues into the platform. With this ordering:

  • A brand-new repository's first-ever sync creates the Repository row and imports zero Issue rows. There is nothing for a sponsor to post a bounty against.
  • Every subsequent re-sync also imports zero issues, because each call re-advances lastSyncedAt to the moment just before it queries.
  • The failure is completely silent and looks like success: the route returns { repository, synced: 0 } with HTTP 201, and the log line reads Synced 0 issues for owner/repo (page 1), which is indistinguishable from a genuinely up-to-date repository.

Webhook-driven issue updates (handleIssueEvent -> upsertIssueRecord) still work for issues that are already tracked, which masks the problem further: a repository can appear to be partially live while its backlog was never imported at all.

Reproduction / failure path

  1. Point the backend at an empty database and a GITHUB_API_TOKEN with read access to a public repo that has open issues (e.g. 20+).
  2. Authenticate as a MAINTAINER who passes RepoScopeGuard (add the repo to GITHUB_SYNC_ALLOWED_REPOS).
  3. POST /api/github/sync/<owner>/<repo>.
  4. Observed: HTTP 201 with synced: 0, a repositories row created with lastSyncedAt set, and SELECT count(*) FROM issues returning 0. The outbound Octokit request carries since=<the timestamp just written>.
  5. Expected: the repo's issues imported on the first call; on a later call, only issues changed since the previous successful sync.

Note that src/github/github-sync.incremental.spec.ts does not catch this: all of its cases (lines 93-130) call syncIssues directly with a hand-constructed Repository object and never go through syncRepository, so the ordering bug lives entirely in the untested seam between the two methods.

Suggested fix

Capture the previous watermark before overwriting it, and only advance the stored value after the page actually succeeds (which also addresses #560's concern):

const previousSyncedAt = repository?.lastSyncedAt ?? null;

const attrs = { /* ... no lastSyncedAt here ... */ };
repository = repository ? merge(...) : create(...);
repository = await this.repositoryRepo.save(repository);

const result = await this.syncIssues(
  { ...repository, lastSyncedAt: previousSyncedAt } as Repository,
  owner, repo, page,
);

// Only now, and only when the page completed without an interruption and
// there is no further page to walk, advance the watermark:
if (!result.nextPage) {
  await this.repositoryRepo.update({ id: repository.id }, { lastSyncedAt: syncStartedAt });
}

Using the timestamp taken before the first API call (syncStartedAt) rather than new Date() after the fact also avoids silently skipping issues that changed while the sync was running. Add a github-sync.service.spec.ts case that calls syncRepository end-to-end against a mocked Octokit and asserts the since parameter equals the stored lastSyncedAt from the previous run (and is absent on the very first run) — the ordering, not just syncIssues in isolation.

Dominant language
TypeScript
Stars
2
Forks
90
Avg merge
31m
Merged PRs (30d)
20

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MergeFi/backend

All issues in MergeFi/backend

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.