[Regression on closed #559] syncRepository still writes lastSyncedAt=now before syncIssues uses it as `since`, so every sync imports zero issues

Aperta
  • Qualcosa non va?
#727 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
67/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
github, typescript
Ambito
api, backend, databases

Direzione di ricerca

Read src/github/github-sync.service.ts, focusing on syncRepository() and syncIssues(), then review src/github/github-sync.incremental.spec.ts to see the existing incremental sync coverage. Add an end-to-end service test that checks the since parameter uses the previous successful sync time, and is absent on the first sync. Done when the regression is covered and the sync imports issues on its first run.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Location

  • src/github/github-sync.service.ts:143-164 — GithubSyncService.syncRepository()
  • src/github/github-sync.service.ts:195-224 — GithubSyncService.syncIssues()

Problem

syncRepository stamps the repository row with lastSyncedAt: new Date() and persists it before calling syncIssues, and syncIssues then reads that very field to build GitHub's since filter.

In syncRepository:

const attrs = {
  githubRepoId: String(repoData.id),
  ...
  lastSyncedAt: new Date(),        // line 153
};

repository = repository
  ? this.repositoryRepo.merge(repository, attrs)
  : this.repositoryRepo.create(attrs);
repository = await this.repositoryRepo.save(repository);   // line 159
...
const result = await this.syncIssues(repository, owner, repo, page);  // line 164

In syncIssues:

const since =
  page === 1 && repository.lastSyncedAt instanceof Date
    ? repository.lastSyncedAt.toISOString()      // line 207 — this is "now"
    : undefined;

The in-memory repository object handed to syncIssues carries the timestamp that was created microseconds earlier, so since is always approximately the current instant. GET /repos/{owner}/{repo}/issues?since=<now> returns only issues updated after that instant — in practice an empty array.

This is the exact defect described by closed issue #559, and the closely related #560 (lastSyncedAt advancing before the sync's outcome is known). The incremental-sync behaviour added for #315 is intact; what regressed is the ordering in the caller.

Why it matters

POST /api/github/sync/:owner/:repo is the only path that imports issues into the platform. With this ordering:

  • A brand-new repository's first-ever sync creates the Repository row and imports zero Issue rows. There is nothing for a sponsor to post a bounty against.
  • Every subsequent re-sync also imports zero issues, because each call re-advances lastSyncedAt to the moment just before it queries.
  • The failure is completely silent and looks like success: the route returns { repository, synced: 0 } with HTTP 201, and the log line reads Synced 0 issues for owner/repo (page 1), which is indistinguishable from a genuinely up-to-date repository.

Webhook-driven issue updates (handleIssueEvent -> upsertIssueRecord) still work for issues that are already tracked, which masks the problem further: a repository can appear to be partially live while its backlog was never imported at all.

Reproduction / failure path

  1. Point the backend at an empty database and a GITHUB_API_TOKEN with read access to a public repo that has open issues (e.g. 20+).
  2. Authenticate as a MAINTAINER who passes RepoScopeGuard (add the repo to GITHUB_SYNC_ALLOWED_REPOS).
  3. POST /api/github/sync/<owner>/<repo>.
  4. Observed: HTTP 201 with synced: 0, a repositories row created with lastSyncedAt set, and SELECT count(*) FROM issues returning 0. The outbound Octokit request carries since=<the timestamp just written>.
  5. Expected: the repo's issues imported on the first call; on a later call, only issues changed since the previous successful sync.

Note that src/github/github-sync.incremental.spec.ts does not catch this: all of its cases (lines 93-130) call syncIssues directly with a hand-constructed Repository object and never go through syncRepository, so the ordering bug lives entirely in the untested seam between the two methods.

Suggested fix

Capture the previous watermark before overwriting it, and only advance the stored value after the page actually succeeds (which also addresses #560's concern):

const previousSyncedAt = repository?.lastSyncedAt ?? null;

const attrs = { /* ... no lastSyncedAt here ... */ };
repository = repository ? merge(...) : create(...);
repository = await this.repositoryRepo.save(repository);

const result = await this.syncIssues(
  { ...repository, lastSyncedAt: previousSyncedAt } as Repository,
  owner, repo, page,
);

// Only now, and only when the page completed without an interruption and
// there is no further page to walk, advance the watermark:
if (!result.nextPage) {
  await this.repositoryRepo.update({ id: repository.id }, { lastSyncedAt: syncStartedAt });
}

Using the timestamp taken before the first API call (syncStartedAt) rather than new Date() after the fact also avoids silently skipping issues that changed while the sync was running. Add a github-sync.service.spec.ts case that calls syncRepository end-to-end against a mocked Octokit and asserts the since parameter equals the stored lastSyncedAt from the previous run (and is absent on the very first run) — the ordering, not just syncIssues in isolation.

Lingua principale
TypeScript
Stelle
2
Fork
90
Merge medio
31m
PR unite (30g)
20

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di MergeFi/backend

Tutte le issue di MergeFi/backend

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.