apache/seatunnel

[Feature] Decouple SeaTunnel Client and Server SEATUNNEL_HOME

Open

#10,167 创建于 2025年12月8日

在 GitHub 查看
 (4 评论) (0 反应) (1 负责人)Java (1,432 fork)batch import
help wanted

仓库指标

Star
 (6,897 star)
PR 合并指标
 (平均合并 13天 21小时) (30 天内合并 143 个 PR)

描述

1. Current Problem

Today SeaTunnel assumes that client and server nodes use the same SEATUNNEL_HOME path. This creates deployment inflexibility:

  • Client : /home/user/seatunnel-client
  • Server : /opt/seatunnel-server

When paths differ, code breaks because:

  • Client may send its absolute paths to server
  • Server expects files at locations that don't exist
  • Ops teams forced to use identical paths everywhere

2. Goal

Allow client and server to use different SEATUNNEL_HOME values, while each maintains the standard directory structure internally:

Client: /home/user/seatunnel-client/
  ├── bin/
  ├── connectors/
  ├── config/
  └── lib/

Server: /opt/seatunnel-server/
  ├── bin/
  ├── connectors/
  ├── config/
  └── lib/

Key principle: Each process resolves paths based on its own SEATUNNEL_HOME, not paths from other nodes.


3. Requirements

3.1 Core Requirements

  1. Client and server can use different root paths
  2. Internal structure (connectors/, config/, lib/) remains fixed relative to SEATUNNEL_HOME
  3. Client only sends logical job configs (connector names, not absolute paths) to server
  4. Each side loads resources from its own directories
  5. Full backward compatibility: existing deployments (same path everywhere) work unchanged

4. Implementation Approach

4.1 Core Principle

Each process (client/server) resolves paths independently based on its own SEATUNNEL_HOME environment variable.

4.2 Key Changes

  1. Unified Path Resolution

    • Create a PathResolver utility that reads SEATUNNEL_HOME and resolves standard subdirectories
    • Replace all hardcoded path concatenations with calls to PathResolver
  2. Client-Server Communication

    • Client sends logical references (e.g., connector name "kafka-source") in job configs
    • Server resolves connector from its own $SEATUNNEL_HOME/connectors/ directory
    • Never pass absolute filesystem paths from client to server
  3. Backward Compatibility

    • If SEATUNNEL_HOME is same everywhere (current setup), behavior unchanged
    • No configuration file changes needed

4.3 What Needs Fixing

Identify and fix code that:

  • Assumes client and server have same SEATUNNEL_HOME
  • Passes absolute paths from client to server (e.g., in job submission)
  • Hardcodes path concatenation like System.getenv("SEATUNNEL_HOME") + "/connectors"

贡献者指南