[Bug][GitHub] Closed (unmarked) Pull Requests are re-fetched on every run

Start with plugins/github/tasks/pr_collector.go, where the issue's stack trace points to pull request collection, and trace how incremental collection selects closed, unmerged PRs for individual fetches. Compare the selection with the reported github_updated_at and the API's updated_at behavior. Done means unchanged closed PRs are skipped, while PRs that change or reopen are collected; the issue names no test file, so locate the relevant collector tests first.

Search before asking

• I had searched in the issues and found no similar issues.

What happened

One of my repo failed last night pipeline on the Collect Pull Requests task on a PR that had been closed (not merged) for more than a month and had not been updated since. For the story GitHub returned a 422 on it.

That made me take a deeper look as there was no reason to re-fetch that PR.
it looks like every closed but not merged PR is being re-fetched on every incremental run. On my instance that's about ~15k closed PRs.

For example, one of the oldest PR in my DB was closed in August 2019, has not been updated since, however at every run:

SELECT number, state, merged, github_created_at, closed_at, github_updated_at
FROM _tool_github_pull_requests
WHERE state = 'closed' AND merged = 0 AND merged_at IS NULL
ORDER BY github_updated_at ASC LIMIT 1;

number,state,merged,github_created_at,closed_at,github_updated_at
29,closed,0,2019-08-02 05:39:12,2019-08-08 08:31:19,2019-08-08 08:31:19

SELECT id, created_at, url FROM _raw_github_api_pull_requests
WHERE url LIKE '%/[REPOSITORY]/pulls/29'
ORDER BY created_at DESC LIMIT 7;

id,created_at,url
5014408,2026-10-19 00:27:23,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4992046,2026-10-18 00:27:07,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4969613,2026-10-17 00:27:19,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4947192,2026-10-16 00:27:38,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4924806,2026-10-15 00:27:32,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4902464,2026-10-14 00:27:21,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29
4880192,2026-10-13 00:27:17,https://api.github.com/repos/[ORGANIZATION]/[REPOSITORY]/pulls/29

183 identical fetches over the last 182 days of raw data we retain, for a PR untouched for 7 years.

Sanitized error:

subtask Collect Pull Requests ended unexpectedly
caused by: Retry exceeded 3 times calling repos/[ORGANIZATION]/[REPOSITORY]/pulls/29.
The last error was: Http DoAsync error calling [method:GET path:repos/[ORGANIZATION]/[REPOSITORY]/pulls/29 query:map[]].
Response: {"message":"The request could not be processed because too many files changed","documentation_url":"https://docs.github.com/rest/pulls/pulls#get-a-pull-request","status":"422"} (422)

attached stack trace
  -- stack trace:
  | github.com/apache/devlake/core/runner.RunPluginSubTasks
  |     /app/core/runner/run_task.go:341
  | [...repeated from below...]
Wraps: (2) subtask Collect Pull Requests ended unexpectedly
Wraps: (3) attached stack trace
  -- stack trace:
  | github.com/apache/devlake/helpers/pluginhelper/api.(*WorkerScheduler).WaitAsync
  |     /app/helpers/pluginhelper/api/worker_scheduler.go:173
  | github.com/apache/devlake/helpers/pluginhelper/api.(*ApiCollector).Execute
  |     /app/helpers/pluginhelper/api/api_collector.go:206
  | github.com/apache/devlake/helpers/pluginhelper/api.(*StatefulApiCollector).Execute
  |     /app/helpers/pluginhelper/api/api_collector_stateful.go:97
  | github.com/apache/devlake/plugins/github/tasks.CollectApiPullRequests
  |     /app/plugins/github/tasks/pr_collector.go:141
  | github.com/apache/devlake/core/runner.runSubtask
  |     /app/core/runner/run_task.go:433
  | github.com/apache/devlake/core/runner.RunPluginSubTasks
  |     /app/core/runner/run_task.go:338
  | github.com/apache/devlake/core/runner.RunPluginTask
  |     /app/core/runner/run_task.go:173
  | github.com/apache/devlake/core/runner.RunTask
  |     /app/core/runner/run_task.go:147
  | github.com/apache/devlake/server/services.runTaskStandalone
  |     /app/server/services/task_runner.go:114
  | github.com/apache/devlake/server/services.RunTasksStandalone.func1
  |     /app/server/services/task.go:187
  | runtime.goexit
  |     /usr/local/go/src/runtime/asm_amd64.s:1771
Wraps: (4) Retry exceeded 3 times calling repos/[ORGANIZATION]/[REPOSITORY]/pulls/29. ...
Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.leafError

### What do you expect to happen

A PR should only be re-fetched individually when it has actually changed since it was last collected, else in a few years time will be fetching way more closed than opened PR.

Reopening, pushing, commenting, reviewing, merging or closing a PR all bump `updated_at`, so if a closed PR gets re-opened Devlake should pick it up.


### How to reproduce

1. Collect a repository that has closed-unmerged PRs.
2. Run incremental collection again.
3. Observe one `GET /repos/{org}/{repo}/pulls/{number}` per open **and** per closed-unmerged
   PR in the database, on every run, including PRs untouched for years:
   ```sql
   SELECT COUNT(*) FROM _tool_github_pull_requests
   WHERE state != 'closed' OR merged = 0;

Anything else

Happens at every collection, and I assume collection time will increase the more unmerged closed PR there are.
Also DB size will become bloated with duplicated data seeing that each non-updated run stores the same payload in _raw_github_api_pull_requests.

Hope I didn't make obvious mistakes in that analysis.

Version

v1.0.3-beta18

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct