drainer is not compatiable with placement rules
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 30/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Ferma
- Ambito
- databases, distributed-systems
Direzione di ricerca
Inizia in drainer/schema.go, all'interno di handleDDL, dove lo stack trace segnala un errore per il job create placement policy con table ID 0; esamina come syncer.go raggiunge questo percorso. Riproduci il problema con i comandi SQL di TiUP e placement riportati nella segnalazione, quindi verifica che Drainer continui a replicare e gestisca questa DDL senza andare in crash.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Bug Report
Please answer these questions before submitting your issue. Thanks!
- What did you do?
a. Use tiup to start a tidb cluster and use drainer to replicate to MySQL
tiup playground v5.3.0 --monitor=false --tiflash 0 --pump 1 --drainer 1
b. Run placement SQL in TiDB
mysql> set @@global.tidb_enable_alter_placement=1;
Query OK, 0 rows affected (0.01 sec)
mysql> CREATE PLACEMENT POLICY x1 FOLLOWERS=4;
Query OK, 0 rows affected (0.18 sec)
mysql> CREATE TABLE T1 (ID INT) PLACEMENT POLICY =`x1`;
Query OK, 0 rows affected (0.23 sec)
c. Check drainer's log.
[2021/12/28 14:41:22.771 +08:00] [ERROR] [main.go:69] ["start drainer server failed"] [error="handle ddl job ID:57, Type:create placement policy, State:synced, SchemaState:public, SchemaID:1, TableID:0, RowCount:0, ArgLen:0, start time: 2021-12-28 14:41:21.591 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0 failed, the schema info: {\n\t\t\"hasImplicitCol\": false,\n\t\t\"schemaMetaVersion\": 0,\n\t\t\"schemaNameToID\": {\n\t\t\t\"mysql\": 3,\n\t\t\t\"test\": 1\n\t\t},\n\t\t\"tableIDToName\": {\n\t\t\t\"11\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"tables_priv\"\n\t\t\t},\n\t\t\t\"13\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"columns_priv\"\n\t\t\t},\n\t\t\t\"15\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"GLOBAL_VARIABLES\"\n\t\t\t},\n\t\t\t\"17\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"tidb\"\n\t\t\t},\n\t\t\t\"19\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"help_topic\"\n\t\t\t},\n\t\t\t\"21\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_meta\"\n\t\t\t},\n\t\t\t\"23\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_histograms\"\n\t\t\t},\n\t\t\t\"25\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_buckets\"\n\t\t\t},\n\t\t\t\"27\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"gc_delete_range\"\n\t\t\t},\n\t\t\t\"29\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"gc_delete_range_done\"\n\t\t\t},\n\t\t\t\"31\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_feedback\"\n\t\t\t},\n\t\t\t\"33\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"role_edges\"\n\t\t\t},\n\t\t\t\"35\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"default_roles\"\n\t\t\t},\n\t\t\t\"37\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"bind_info\"\n\t\t\t},\n\t\t\t\"39\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_top_n\"\n\t\t\t},\n\t\t\t\"41\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"expr_pushdown_blacklist\"\n\t\t\t},\n\t\t\t\"43\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"opt_rule_blacklist\"\n\t\t\t},\n\t\t\t\"45\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_extended\"\n\t\t\t},\n\t\t\t\"47\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"schema_index_usage\"\n\t\t\t},\n\t\t\t\"49\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_fm_sketch\"\n\t\t\t},\n\t\t\t\"5\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"user\"\n\t\t\t},\n\t\t\t\"51\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"global_grants\"\n\t\t\t},\n\t\t\t\"53\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"capture_plan_baselines_blacklist\"\n\t\t\t},\n\t\t\t\"55\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"column_stats_usage\"\n\t\t\t},\n\t\t\t\"7\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"global_priv\"\n\t\t\t},\n\t\t\t\"9\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"db\"\n\t\t\t}\n\t\t}\n\t}: table 0 not found"] [errorVerbose="table 0 not found\ngithub.com/pingcap/errors.NotFoundf\n\t/Users/pingcap/gopkg/pkg/mod/github.com/pingcap/[email protected]/juju_adaptor.go:117\ngithub.com/pingcap/tidb-binlog/drainer.(*Schema).handleDDL\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/schema.go:476\ngithub.com/pingcap/tidb-binlog/drainer.(*Schema).handlePreviousDDLJobIfNeed\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/schema.go:297\ngithub.com/pingcap/tidb-binlog/drainer.(*Syncer).run\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/syncer.go:427\ngithub.com/pingcap/tidb-binlog/drainer.(*Syncer).Start\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/syncer.go:133\ngithub.com/pingcap/tidb-binlog/drainer.(*Server).Start.func4\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/server.go:290\ngithub.com/pingcap/tidb-binlog/drainer.(*taskGroup).start.func1\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb-binlog/drainer/util.go:72\nruntime.goexit\n\t/usr/local/go1.16.4/src/runtime/asm_amd64.s:1371\nhandle ddl job ID:57, Type:create placement policy, State:synced, SchemaState:public, SchemaID:1, TableID:0, RowCount:0, ArgLen:0, start time: 2021-12-28 14:41:21.591 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0 failed, the schema info: {\n\t\t\"hasImplicitCol\": false,\n\t\t\"schemaMetaVersion\": 0,\n\t\t\"schemaNameToID\": {\n\t\t\t\"mysql\": 3,\n\t\t\t\"test\": 1\n\t\t},\n\t\t\"tableIDToName\": {\n\t\t\t\"11\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"tables_priv\"\n\t\t\t},\n\t\t\t\"13\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"columns_priv\"\n\t\t\t},\n\t\t\t\"15\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"GLOBAL_VARIABLES\"\n\t\t\t},\n\t\t\t\"17\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"tidb\"\n\t\t\t},\n\t\t\t\"19\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"help_topic\"\n\t\t\t},\n\t\t\t\"21\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_meta\"\n\t\t\t},\n\t\t\t\"23\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_histograms\"\n\t\t\t},\n\t\t\t\"25\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_buckets\"\n\t\t\t},\n\t\t\t\"27\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"gc_delete_range\"\n\t\t\t},\n\t\t\t\"29\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"gc_delete_range_done\"\n\t\t\t},\n\t\t\t\"31\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_feedback\"\n\t\t\t},\n\t\t\t\"33\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"role_edges\"\n\t\t\t},\n\t\t\t\"35\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"default_roles\"\n\t\t\t},\n\t\t\t\"37\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"bind_info\"\n\t\t\t},\n\t\t\t\"39\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_top_n\"\n\t\t\t},\n\t\t\t\"41\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"expr_pushdown_blacklist\"\n\t\t\t},\n\t\t\t\"43\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"opt_rule_blacklist\"\n\t\t\t},\n\t\t\t\"45\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_extended\"\n\t\t\t},\n\t\t\t\"47\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"schema_index_usage\"\n\t\t\t},\n\t\t\t\"49\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"stats_fm_sketch\"\n\t\t\t},\n\t\t\t\"5\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"user\"\n\t\t\t},\n\t\t\t\"51\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"global_grants\"\n\t\t\t},\n\t\t\t\"53\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"capture_plan_baselines_blacklist\"\n\t\t\t},\n\t\t\t\"55\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"column_stats_usage\"\n\t\t\t},\n\t\t\t\"7\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"global_priv\"\n\t\t\t},\n\t\t\t\"9\": {\n\t\t\t\t\"db-name\": \"mysql\",\n\t\t\t\t\"tbl-name\": \"db\"\n\t\t\t}\n\t\t}\n\t}"]
-
What did you expect to see?
Drainer replicates data successfully and filter this kind of DDL correctly. -
What did you see instead?
Drainer crashed. -
Please provide the relate downstream type and version of drainer.
(rundrainer -Vin terminal to get drainer's version)
[2021/12/28 14:39:45.047 +08:00] [INFO] [version.go:50] ["Welcome to Drainer"] ["Release Version"=v5.3.0] ["Git Commit Hash"=c032e8ba9b524a6a698475a76cc2926db17b3cad] ["Build TS"="2021-11-16 11:53:46"] ["Go Version"=go1.16.4] ["Go OS/Arch"=darwin/amd64]
- Lingua principale
- Go
- Stelle
- 291
- Fork
- 131
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pingcap/tidb-binlog
-
Drainer's memory usage exponentially increases when filtering too many DELETE events, leading to OOMApertaaffects-5.4 affects-6.5
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
pingcap/tidb-binlog#1317 · 2 commenti ·
-
affects-6.5 affects-7.1 affects-7.5 affects-8.1 type/bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 48/100
pingcap/tidb-binlog#1312 ·
-
remove arbiterForse già presa @lance6716 l’ha presa 838 giorni fa. Apertafeature-request
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
pingcap/tidb-binlog#1310 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
pingcap/tidb-binlog#1267 · 1 commento ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 45/100
pingcap/tidb-binlog#1262 ·
Tutte le issue di pingcap/tidb-binlog
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
github/github-mcp-server#3865 ·
I maintainer di solito rispondono entro 2 giorni
-
kind/features
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
TencentBlueKing/bk-nodemgr#4017 ·
I maintainer di solito rispondono entro 1 giorno
-
ssa: wait errors drop the kstatus message, so health check failures don't say whyForse già presa @schdief l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
equinor/radix-operator#2066 ·
I maintainer di solito rispondono entro 2 giorni