apache/seatunnel
Ver no GitHub[Connector] Capture log files using streaming mode, but the process stops after capturing a file.
Open
#10.225 aberto em 22 de dez. de 2025
help wanted
Métricas do repositório
- Stars
- (6.897 stars)
- Métricas de merge de PR
- (Mesclagem média 13d 21h) (143 fundiu PRs em 30d)
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
Collect log files using streaming mode, but the process stops after collecting one file.
Even if data is appended to the file, it still doesn't work.
SeaTunnel Version
2.3.12
SeaTunnel Config
env {
job.mode = "STREAMING"
}
source {
LocalFile {
path = "/data/app/message_log/message-service-center.log"
file_format_type = "text"
schema {
fields {
raw_line = "string"
}
}
file_read_mode = "rolling"
monitoring_interval_ms = 10000
poll_interval = 10000
plugin_output = "original_log_line"
}
}
transform {
Split {
plugin_input = "original_log_line"
separator = "发送消息成功:"
split_field = "raw_line"
output_fields = [first_name, second_name]
plugin_output = "split_log_parts"
}
Filter {
plugin_input = "split_log_parts"
plugin_output = "filter_log_parts"
include_fields = [second_name]
}
JsonPath {
plugin_input = "filter_log_parts"
row_error_handle_way = FAIL
columns = [
{
"src_field" = "second_name"
"path" = "$.id"
"dest_field" = "id"
"column_error_handle_way" = "SKIP_ROW"
},
{
"src_field" = "second_name"
"path" = "$.receiver"
"dest_field" = "receiver"
"column_error_handle_way" = "SKIP_ROW"
}
]
plugin_output = "json_parsed_data"
}
FieldMapper {
plugin_input = "json_parsed_data"
plugin_output = "final_parsed_data"
field_mapper = {
id = id
receiver = receiver
}
}
}
Running Command
nohup bin/seatunnel.sh --config ../job/test_file_to_mysql.conf -m local >seatunnel.log 2>&1 &
Error Exception
don't report an error, but it stops.
Zeta or Flink or Spark Version
No response
Java or Scala Version
No response
Screenshots
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct