[DSIP-100][Api Worker] Generic Third-party System API Connector for REST-based Task Scheduling
维护者通常 1 天内回复
评估
这个 Issue 还没有评估数据。
描述
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
Propose a Generic Third-party System API Connector to enable DolphinScheduler to schedule tasks on any external system via REST APIs.
Currently, DS lacks a unified way to integrate with systems like SeaTunnel, Flink Session, or custom platforms that expose task control via RESTful interfaces. This feature would provide a configurable framework (managing endpoints for auth, task list, start, status, stop) instead of requiring custom plugins for each system, greatly expanding DS's scheduling capabilities.
Use case
As a DS administrator or user,
I want to register an external system (e.g., company's customized SeaTunnel system) or other by providing its base URL, authentication, and specific API endpoints (to list, start, check status, and stop tasks),
So that I can easily select and schedule that system's tasks directly within a DolphinScheduler workflow.
The expected behavior is:
1、In the DS UI, I can add a new "Third-party System" config for our SeaTunnel server's API.
2、When building a workflow, I can choose the "Generic API Task" type, select the registered "SeaTunnel" system, and see a dropdown list of available jobs fetched from its API.
3、Upon selecting a job and running the workflow, DS automatically calls the start API, polls the status API until completion, and reflects the final status (success/failure) in DS.
4、I can also manually stop the task from DS, which triggers the stop API call to the external system.
This will allow us to use DS as a unified scheduler for many API-based tools without waiting for specific plugin development.
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
- 主要语言
- Java
- 星标
- 14.5k
- 派生
- 5.1k
- 平均合并
- 2 天 12 小时
- 30 天内合并 PR
- 26
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/dolphinscheduler 的其他 Issue
-
[Bug][Master] Lost terminal ACK leaves task executors waiting after workflow finalization可能已有人在做 @csurong 今天认领。 未关闭backend bug
apache/dolphinscheduler#18735 · 已指派 1 人 ·
维护者通常 1 天内回复
-
[Improvement][Worker-SqlTask] Make the default query row limit (QUERY_LIMIT = 10000) configurable可能已有人在做 @njnu-seafish 于 10 天前认领。 未关闭backend improvement UI
apache/dolphinscheduler#18702 · 已指派 1 人 ·
维护者通常 1 天内回复
-
[Improvement][APIServer][3.4.3] Task Instance page data loading time out.可能已有人在做 @karthikeya-tummala 于 10 天前认领。 未关闭backend good first issue improvement
apache/dolphinscheduler#18700 · 6 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
bug need more information
难度 4/5 3-5 天 新手友好度 42/100
apache/dolphinscheduler#18698 · 5 条评论 ·
维护者通常 1 天内回复
-
[Bug][Task] DinkyTask spuriously interrupts the worker thread, permanently leaking worker slots`可能已有人在做 @sql-wj 于 11 天前认领。 未关闭backend bug
apache/dolphinscheduler#18697 · 已指派 1 人 ·
维护者通常 1 天内回复
查看 apache/dolphinscheduler 的全部 Issue
相似的 Issue
-
code-maintenance
难度 2/5 1-3 小时 新手友好度 77/100
维护者通常 1 天内回复
-
bug priority: medium Review effort [1-5]: 3 status: needs-triage type: bug
难度 2/5 1-3 小时 新手友好度 82/100
carlos-emr/carlos#5258 ·
维护者通常 1 天内回复
-
bug: author create/update with an unknown avatarId returns 404 instead of 400 UNKNOWN_REFERENCE可能已有人在做 @Vinu2111 今天认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 74/100
atomityhq/blog-cms#59 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
dev/release: verify-release-candidate.sh runs 0 Java tests with Debian's Maven可能已有人在做 @kou 今天认领。 未关闭Type: bug
难度 2/5 1-3 小时 新手友好度 72/100
apache/arrow-adbc#4947 ·
维护者通常 1 天内回复