apache/seatunnel

[Feature][connector] http connector cursor type pageing not check the number of returned items is less than batchSize？

Open

#10,929 建立於 2026年5月22日

在 GitHub 查看
 (4 留言) (0 反應) (0 負責人)Java (6,897 star) (1,432 fork)batch import
bughelp wanted

描述

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

in HttpSourceReader.java 406-407, there are only one kind check is "noMoreElementFlag = Strings.isNullOrEmpty(newCursor)" why not check datasize ? some kind of system return the last same cursor when data is over!

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

貢獻者指南

技術棧
javarest api
領域
dataapi
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
2
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-3 hours
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
fresh
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
JavaHTTP paginationSeaTunnel connector basics
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
75
研究方向
Examine HttpSourceReader.java around lines 406-407 where 'noMoreElementFlag' is set. Understand the current cursor based pagination logic. Determine when the API returns the same cursor when data is exhausted. Implement a check comparing the number of returned items to the configured batchSize to detect end of data. Consider adding unit tests for edge cases.