apache/seatunnel

[Connector] Capture log files using streaming mode, but the process stops after capturing a file.

Open

#10,225 创建于 2025年12月22日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Java (1,432 fork)batch import
help wanted

仓库指标

Star
 (6,897 star)
PR 合并指标
 (平均合并 13天 21小时) (30 天内合并 143 个 PR)

描述

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

Collect log files using streaming mode, but the process stops after collecting one file.

Even if data is appended to the file, it still doesn't work.

SeaTunnel Version

2.3.12

SeaTunnel Config

env {
     job.mode = "STREAMING"
}

source {
  LocalFile {
    path = "/data/app/message_log/message-service-center.log"
    file_format_type = "text"
    schema {
      fields {
        raw_line = "string"
      }
    }
    file_read_mode = "rolling"
    monitoring_interval_ms = 10000
    poll_interval = 10000
    plugin_output = "original_log_line"
  }
}

transform {
     Split {
         plugin_input = "original_log_line"
         separator = "发送消息成功:"
         split_field = "raw_line"
         output_fields = [first_name, second_name]
         plugin_output = "split_log_parts"
       }
     Filter {
         plugin_input = "split_log_parts"
         plugin_output = "filter_log_parts"
         include_fields = [second_name]
       }
    JsonPath {
     plugin_input = "filter_log_parts"
     row_error_handle_way = FAIL
     columns = [
          {
             "src_field" = "second_name"
             "path" = "$.id"
             "dest_field" = "id"
             "column_error_handle_way" = "SKIP_ROW"
          },
          {
            "src_field" = "second_name"
            "path" = "$.receiver"
            "dest_field" = "receiver"
            "column_error_handle_way" = "SKIP_ROW"
          }
     ]
     plugin_output = "json_parsed_data"
  }
   FieldMapper {
       plugin_input = "json_parsed_data"
       plugin_output = "final_parsed_data"
       field_mapper = {
           id = id
           receiver = receiver
       }
     }
}

Running Command

nohup bin/seatunnel.sh --config ../job/test_file_to_mysql.conf -m local >seatunnel.log 2>&1 &

Error Exception

 don't report an error, but it stops.

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

贡献者指南