apache/seatunnel

[Feature][connector] http connector cursor type pageing not check the number of returned items is less than batchSize？

Open

#10929 opened on May 22, 2026

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Java (6,897 stars) (1,432 forks)batch import
bughelp wanted

Description

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

in HttpSourceReader.java 406-407, there are only one kind check is "noMoreElementFlag = Strings.isNullOrEmpty(newCursor)" why not check datasize ? some kind of system return the last same cursor when data is over!

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Tech stack
javarest api
Domain
dataapi
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
JavaHTTP paginationSeaTunnel connector basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75
Research direction
Examine HttpSourceReader.java around lines 406-407 where 'noMoreElementFlag' is set. Understand the current cursor based pagination logic. Determine when the API returns the same cursor when data is exhausted. Implement a check comparing the number of returned items to the configured batchSize to detect end of data. Consider adding unit tests for edge cases.