[Bug][Asana] Story, subtask and tag collectors stop at 100 per task; task and user collectors restart on a full last page

Ouverte
  • Quelque chose ne va pas ?
#9,365 0 commentaires 0 réactions 1 personne assignée Voir sur GitHub

Les mainteneurs répondent en général sous 2 jours

@ShivamSharma43 y travaille déjà.

Depuis le 10/11/2026.

  • #9366 par @ShivamSharma43 — ouverte

Évaluation

Cette issue n'a pas encore été évaluée.

Description

Search before asking
  • I had searched in the issues and found no similar issues.
What happened

The Asana collectors don't page through list endpoints correctly.

  1. collectStory, collectSubtask and collectTag (plugins/asana/tasks/{story,subtask,tag}_collector.go) send limit=100 but have no PageSize and no GetNextPageCustomData, so the ApiCollector makes one request per task and never sends offset. Only the first 100 stories, subtasks and tags of each task are collected. Stories include system events (assignments, section moves, due date changes), so busy tasks go past 100.

  2. collectTask and collectUser do page with offset, but their GetNextPageCustomData returns nil, nil when next_page is null:

if resp.NextPage != nil && resp.NextPage.Offset != "" {
	return resp.NextPage.Offset, nil
}
return nil, nil

nil, nil doesn't end the collection. The collector sets CustomData to nil and requests the next page, and the query then has no offset, so it asks for page 1 again. When the last page is full (100 items) and Asana returns "next_page": null, the collector keeps re-collecting the project from the start and saving the same raw rows again.

From the Asana docs (https://developers.asana.com/docs/pagination): limit must be between 1 and 100, and "If there are no more pages available, next_page will be null and no offset will be provided."

What do you expect to happen

All five collectors follow next_page.offset and stop when next_page is null.

How to reproduce
  • Stories: collect a project with a task that has more than 100 stories. _raw_asana_stories has 100 rows for that task.
  • Tasks: the GetNextPageCustomData above returns nil, nil for a full page with "next_page": null, and fetchPagesSequentially in helpers/pluginhelper/api/api_collector.go only stops on a short page or ErrFinishCollect.
Anything else

Proposed fix (I have a PR ready): a shared getNextAsanaPageOffset that returns api.ErrFinishCollect when next_page is null, and a setAsanaPageQuery that sets limit and offset, used by the task, user, story, subtask and tag collectors.

Version

main

Are you willing to submit PR?
  • Yes I am willing to submit a PR!
Code of Conduct
Langage dominant
Go
Étoiles
3.2k
Forks
840
Merge moyen
1 j 21 h
PR mergées (30 j)
104

Préparer son environnement

Ouvrir dans Codespaces

Lance le conteneur de développement du projet dans votre navigateur, avec votre propre compte GitHub.

  • Aucun Dockerfile ni fichier Docker Compose
  • Propose un modèle de pull request
  • Aucun guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de apache/devlake

Toutes les issues de apache/devlake

Issues similaires

Plus d'issues Go

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.