DTStack/chunjun

[Bug] [sqlservercdc] 任务提交给单机版的flink数据未同步

Open

#1 240 ouverte le 9 sept. 2022

Voir sur GitHub
 (1 commentaire) (0 réactions) (0 assignés)Java (1 668 forks)batch import
bughelp wanted

Métriques du dépôt

Stars
 (3 752 stars)
Métriques de merge PR
 (Aucune PR mergée en 30 j)

Description

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

任务正常提交给flink,任务执行正常,没有任何异常信息,但是数据未同步到postgresql中。 任务配置信息如下: CREATE TABLE source ( id bigint NOT NULL, parent_id bigint NOT NULL, project_no varchar(50) NOT NULL, sales_order varchar(50) NOT NULL, plant_code varchar(50) NULL, product_line varchar(50) NULL, sales_order_item varchar(50) NOT NULL, bay_id varchar(50) NULL, order_no varchar(50) NOT NULL, order_status int NULL, order_level int NOT NULL, sap_order_status varchar(50) NULL, material_no varchar(50) NOT NULL, material_descr varchar(255) NULL, final_material varchar(50) NULL, init_production_center varchar(50) NULL, final_production_center varchar(50) NULL, module_code varchar(50) NULL, material_no_fellow varchar(50) NULL, order_no_fellow varchar(50) NULL, item_component_list varchar(50) NULL, reservation varchar(50) NULL, reservation_item varchar(50) NULL, material_picking1_times int NULL, material_picking2_times int NULL, first_date timestamp NULL, basic_start_time timestamp NULL, basic_finish_time timestamp NULL, product_start_time timestamp NULL, mes_plan_start_time timestamp NULL, mes_plan_end_time timestamp NULL, mes_task_exec_time timestamp NULL, mes_plan_u_time timestamp NULL, is_deleted int NULL, is_modify int NULL, create_user varchar(50) NULL, update_user varchar(50) NULL, create_time timestamp NULL, update_time timestamp NOT NULL, PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'connector' = 'sqlservercdc-x' ,'username' = 'sa' ,'password' = 'sa@123' ,'cat' = 'insert,delete,update' ,'url' = 'jdbc:sqlserver://192.168.8.212:1433;databaseName=CDCTest' ,'table' = 'dbo.ETO_PDM_Header' ,'timestamp-format.standard' = 'SQL' ,'database' = 'CDCTest' ,'poll-interval' = '1000' );

CREATE TABLE sink ( id numeric NOT NULL, parent_id numeric NOT NULL, project_no varchar(50) NOT NULL, sales_order varchar(50) NOT NULL, plant_code varchar(50) NULL, product_line varchar(50) NULL, sales_order_item varchar(50) NOT NULL, bay_id varchar(50) NULL, order_no varchar(50) NOT NULL, order_status int NULL, order_level int NOT NULL, sap_order_status varchar(50) NULL, material_no varchar(50) NOT NULL, material_descr varchar(255) NULL, final_material varchar(50) NULL, init_production_center varchar(50) NULL, final_production_center varchar(50) NULL, module_code varchar(50) NULL, material_no_fellow varchar(50) NULL, order_no_fellow varchar(50) NULL, item_component_list varchar(50) NULL, reservation varchar(50) NULL, reservation_item varchar(50) NULL, material_picking1_times int NULL, material_picking2_times int NULL, first_date timestamp NULL, basic_start_time timestamp NULL, basic_finish_time timestamp NULL, product_start_time timestamp NULL, mes_plan_start_time timestamp NULL, mes_plan_end_time timestamp NULL, mes_task_exec_time timestamp NULL, mes_plan_u_time timestamp NULL, is_deleted int NULL, is_modify int NULL, create_user varchar(50) NULL, update_user varchar(50) NULL, create_time timestamp NULL, update_time timestamp NULL, PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'connector' = 'postgresql-x', 'url' = 'jdbc:postgresql://192.168.0.116:52345/postgres', 'table-name' = 'siemens2_ETO_PDM_Header', 'username' = 'postgres', 'password' = 'P@stgres123', 'sink.buffer-flush.max-rows' = '1024', 'sink.buffer-flush.interval' = '1000', 'sink.all-replace' = 'true', 'sink.parallelism' = '1' );

insert into sink select * from source u;

任务提交脚本: sh bin/chunjun-standalone.sh -jobName sqlserverCDCToPostgresql -job sqlserver_cdc_pg.sql

提交成功之后,flink运行正常,数据就是没有同步 sqlserver数据库cdc的配置也是按照chunjun文档去配置的

What you expected to happen

数据未同步

How to reproduce

按照问题描述去操作

Anything else

No response

Version

master

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guide contributeur