Migration hangs when --attempt-instant-ddl succeeds

Aberta
  • Algo errado?
#1,795 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 10 dias

@d-bytebase já está trabalhando nisso.

Desde 2/11/2026.

  • #1796 de @d-bytebase — aberto

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
30/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
go, mysql
Domínio
databases

Direção de pesquisa

Start with Migrate() in go/logic/migrator.go, where the instant-DDL branch returns without reading ghostTableMigrated, then follow onChangelogStateEvent and notifyListeners to see where listenersMutex is held during the blocking send. The binlog side is shouldDecodeRowsEvent in the event streamer, reached through BinlogSyncer.Close() in finalCleanup. Done means the instant-DDL success path returns without hanging, with a test covering that short-circuit.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Overview

A migration run with --attempt-instant-ddl against a server that can apply the
ALTER instantly applies it, logs Success! table ... migrated instantly, and then
never returns. gh-ost hangs until it is killed.

Reproduced on master (dd44453f) and on v1.1.11.

Why

Migrate() short-circuits on the instant-DDL path: on success it calls
finalCleanup(), runs the OnSuccess hook and returns, so it never reads
<-mgtr.ghostTableMigrated.

https://github.com/github/gh-ost/blob/dd44453f489bdeb8bee5b93194a0a481e001e598/go/logic/migrator.go#L534-L552

Three goroutines then form a cycle:

  1. The changelog listener parks on the GhostTableMigrated send in
    onChangelogStateEvent. ghostTableMigrated is unbuffered and the
    instant-DDL path never reads it, so the send blocks. It is wrapped in
    base.SendWithContext, which only unblocks on context cancellation — and
    nothing cancels the context here, precisely because Migrate() has not
    returned. The listener is invoked from notifyListeners, which holds
    listenersMutex across the synchronous call, so the mutex stays held for as
    long as the send is parked.

  2. The binlog parser goroutine (go-mysql's BinlogSyncer.onStream) calls
    EventsStreamer.shouldDecodeRowsEvent through RowsEventDecodeFunc and
    blocks taking that same listenersMutex.

  3. BinlogSyncer.Close(), reached from finalCleanup() →
    EventsStreamer.Close(), ends in b.wg.Wait() and waits for that parser
    goroutine, which is never coming back.

Step 2 is what makes the cycle reachable. shouldDecodeRowsEvent was introduced
by #1699 ("Don't parse binlog events of unrelated tables", first released in
v1.1.11); it put listenersMutex on the binlog parser's goroutine, so a listener
parked while holding that mutex now stalls the parser directly, and with it
BinlogSyncer.Close().

--attempt-instant-ddl defaults to off and the short-circuit has no test
coverage, which is presumably why this has not been reported before. We hit it in
a downstream consumer that passes the flag for every migration, where every ALTER
MySQL could apply instantly hung.

Reproduction

Any instant-applicable ALTER on MySQL 8.0 with the flag set, e.g.

gh-ost --host=... --database=test --table=testing \
  --alter='ADD COLUMN author varchar(255)' \
  --attempt-instant-ddl --allow-on-master --execute

The column is added, Success! ... migrated instantly is logged, and the process
does not exit.

Fix

PR to follow: scope listenersMutex to the listener lookup in notifyListeners,
and give ghostTableMigrated a one-slot buffer.

Linguagem predominante
Go
Estrelas
13.6k
Forks
1.4k
Merge médio
9d 19h
PRs com merge (30d)
1

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de github/gh-ost

Todas as issues de github/gh-ost

Issues semelhantes

Mais issues de Go

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.