DTStack/chunjun

[Bug] [sqlservercdc] 任务提交给单机版的flink数据未同步

Open

#1,240 opened on Sep 9, 2022

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Java (3,752 stars) (1,668 forks)batch import
bughelp wanted

Description

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

任务正常提交给flink,任务执行正常,没有任何异常信息,但是数据未同步到postgresql中。 任务配置信息如下: CREATE TABLE source ( id bigint NOT NULL, parent_id bigint NOT NULL, project_no varchar(50) NOT NULL, sales_order varchar(50) NOT NULL, plant_code varchar(50) NULL, product_line varchar(50) NULL, sales_order_item varchar(50) NOT NULL, bay_id varchar(50) NULL, order_no varchar(50) NOT NULL, order_status int NULL, order_level int NOT NULL, sap_order_status varchar(50) NULL, material_no varchar(50) NOT NULL, material_descr varchar(255) NULL, final_material varchar(50) NULL, init_production_center varchar(50) NULL, final_production_center varchar(50) NULL, module_code varchar(50) NULL, material_no_fellow varchar(50) NULL, order_no_fellow varchar(50) NULL, item_component_list varchar(50) NULL, reservation varchar(50) NULL, reservation_item varchar(50) NULL, material_picking1_times int NULL, material_picking2_times int NULL, first_date timestamp NULL, basic_start_time timestamp NULL, basic_finish_time timestamp NULL, product_start_time timestamp NULL, mes_plan_start_time timestamp NULL, mes_plan_end_time timestamp NULL, mes_task_exec_time timestamp NULL, mes_plan_u_time timestamp NULL, is_deleted int NULL, is_modify int NULL, create_user varchar(50) NULL, update_user varchar(50) NULL, create_time timestamp NULL, update_time timestamp NOT NULL, PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'connector' = 'sqlservercdc-x' ,'username' = 'sa' ,'password' = 'sa@123' ,'cat' = 'insert,delete,update' ,'url' = 'jdbc:sqlserver://192.168.8.212:1433;databaseName=CDCTest' ,'table' = 'dbo.ETO_PDM_Header' ,'timestamp-format.standard' = 'SQL' ,'database' = 'CDCTest' ,'poll-interval' = '1000' );

CREATE TABLE sink ( id numeric NOT NULL, parent_id numeric NOT NULL, project_no varchar(50) NOT NULL, sales_order varchar(50) NOT NULL, plant_code varchar(50) NULL, product_line varchar(50) NULL, sales_order_item varchar(50) NOT NULL, bay_id varchar(50) NULL, order_no varchar(50) NOT NULL, order_status int NULL, order_level int NOT NULL, sap_order_status varchar(50) NULL, material_no varchar(50) NOT NULL, material_descr varchar(255) NULL, final_material varchar(50) NULL, init_production_center varchar(50) NULL, final_production_center varchar(50) NULL, module_code varchar(50) NULL, material_no_fellow varchar(50) NULL, order_no_fellow varchar(50) NULL, item_component_list varchar(50) NULL, reservation varchar(50) NULL, reservation_item varchar(50) NULL, material_picking1_times int NULL, material_picking2_times int NULL, first_date timestamp NULL, basic_start_time timestamp NULL, basic_finish_time timestamp NULL, product_start_time timestamp NULL, mes_plan_start_time timestamp NULL, mes_plan_end_time timestamp NULL, mes_task_exec_time timestamp NULL, mes_plan_u_time timestamp NULL, is_deleted int NULL, is_modify int NULL, create_user varchar(50) NULL, update_user varchar(50) NULL, create_time timestamp NULL, update_time timestamp NULL, PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'connector' = 'postgresql-x', 'url' = 'jdbc:postgresql://192.168.0.116:52345/postgres', 'table-name' = 'siemens2_ETO_PDM_Header', 'username' = 'postgres', 'password' = 'P@stgres123', 'sink.buffer-flush.max-rows' = '1024', 'sink.buffer-flush.interval' = '1000', 'sink.all-replace' = 'true', 'sink.parallelism' = '1' );

insert into sink select * from source u;

任务提交脚本: sh bin/chunjun-standalone.sh -jobName sqlserverCDCToPostgresql -job sqlserver_cdc_pg.sql

提交成功之后,flink运行正常,数据就是没有同步 sqlserver数据库cdc的配置也是按照chunjun文档去配置的

What you expected to happen

数据未同步

How to reproduce

按照问题描述去操作

Anything else

No response

Version

master

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide