[Feature][connector] http connector cursor type pageing not check the number of returned items is less than batchSize？

(4 則留言) (0 個反應) (0 位負責人)Java (1,432 個分叉)batch import

bughelp wanted

倉庫指標

星標: (6,897 顆星)
PR 合併指標: (PR 指標待抓取)

描述

Search before asking

I had searched in the feature and found no similar feature requirement.

Description

in HttpSourceReader.java 406-407, there are only one kind check is "noMoreElementFlag = Strings.isNullOrEmpty(newCursor)" why not check datasize ? some kind of system return the last same cursor when data is over!

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

貢獻者指南

研究方向: 檢查 HttpSourceReader.java 檔案第 406-407 行附近的程式碼，了解當前的基於游標的分頁邏輯。確定 `Strings.isNullOrEmpty(newCursor)` 條件是否足以檢測資料結束。考慮加入對返回項目數量小於 batchSize 的檢查，以處理伺服器在資料耗盡後仍然返回相同游標的情況。
技術棧: java
領域: apidata
議題類型: 錯誤
難度: 2
預計時間: 半天
活動狀態: 活躍
清晰度: 大致清晰
前置要求: JavaGit
新手友善度: 70