Migration hangs when --attempt-instant-ddl succeeds

オープン
  • 何か問題がありますか？
#1,795 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 10 日以内に返信

@d-bytebase がすでに取り組んでいます。

2026年11月2日 から。

  • #1796 @d-bytebase による — オープン

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
30/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
go, mysql
領域
databases

調査の方向性

Start with Migrate() in go/logic/migrator.go, where the instant-DDL branch returns without reading ghostTableMigrated, then follow onChangelogStateEvent and notifyListeners to see where listenersMutex is held during the blocking send. The binlog side is shouldDecodeRowsEvent in the event streamer, reached through BinlogSyncer.Close() in finalCleanup. Done means the instant-DDL success path returns without hanging, with a test covering that short-circuit.

索引モデルが issue の本文から書いたものです。

説明

Overview

A migration run with --attempt-instant-ddl against a server that can apply the
ALTER instantly applies it, logs Success! table ... migrated instantly, and then
never returns. gh-ost hangs until it is killed.

Reproduced on master (dd44453f) and on v1.1.11.

Why

Migrate() short-circuits on the instant-DDL path: on success it calls
finalCleanup(), runs the OnSuccess hook and returns, so it never reads
<-mgtr.ghostTableMigrated.

https://github.com/github/gh-ost/blob/dd44453f489bdeb8bee5b93194a0a481e001e598/go/logic/migrator.go#L534-L552

Three goroutines then form a cycle:

  1. The changelog listener parks on the GhostTableMigrated send in
    onChangelogStateEvent. ghostTableMigrated is unbuffered and the
    instant-DDL path never reads it, so the send blocks. It is wrapped in
    base.SendWithContext, which only unblocks on context cancellation — and
    nothing cancels the context here, precisely because Migrate() has not
    returned. The listener is invoked from notifyListeners, which holds
    listenersMutex across the synchronous call, so the mutex stays held for as
    long as the send is parked.

  2. The binlog parser goroutine (go-mysql's BinlogSyncer.onStream) calls
    EventsStreamer.shouldDecodeRowsEvent through RowsEventDecodeFunc and
    blocks taking that same listenersMutex.

  3. BinlogSyncer.Close(), reached from finalCleanup() →
    EventsStreamer.Close(), ends in b.wg.Wait() and waits for that parser
    goroutine, which is never coming back.

Step 2 is what makes the cycle reachable. shouldDecodeRowsEvent was introduced
by #1699 ("Don't parse binlog events of unrelated tables", first released in
v1.1.11); it put listenersMutex on the binlog parser's goroutine, so a listener
parked while holding that mutex now stalls the parser directly, and with it
BinlogSyncer.Close().

--attempt-instant-ddl defaults to off and the short-circuit has no test
coverage, which is presumably why this has not been reported before. We hit it in
a downstream consumer that passes the flag for every migration, where every ALTER
MySQL could apply instantly hung.

Reproduction

Any instant-applicable ALTER on MySQL 8.0 with the flag set, e.g.

gh-ost --host=... --database=test --table=testing \
  --alter='ADD COLUMN author varchar(255)' \
  --attempt-instant-ddl --allow-on-master --execute

The column is added, Success! ... migrated instantly is logged, and the process
does not exit.

Fix

PR to follow: scope listenersMutex to the listener lookup in notifyListeners,
and give ghostTableMigrated a one-slot buffer.

主要言語
Go
スター
13.6k
フォーク
1.4k
平均マージ
9日 19時間
マージ済み PR（30日）
1

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

github/gh-ost のほかの issue

github/gh-ost の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。