Migration hangs when --attempt-instant-ddl succeeds

Abierto
  • ¿Algo no está bien?
#1,795 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 10 días

@d-bytebase ya está trabajando en esto.

Desde el 2/11/2026.

  • #1796 de @d-bytebase — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
go, mysql
Área
databases

Línea de trabajo

Start with Migrate() in go/logic/migrator.go, where the instant-DDL branch returns without reading ghostTableMigrated, then follow onChangelogStateEvent and notifyListeners to see where listenersMutex is held during the blocking send. The binlog side is shouldDecodeRowsEvent in the event streamer, reached through BinlogSyncer.Close() in finalCleanup. Done means the instant-DDL success path returns without hanging, with a test covering that short-circuit.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Overview

A migration run with --attempt-instant-ddl against a server that can apply the
ALTER instantly applies it, logs Success! table ... migrated instantly, and then
never returns. gh-ost hangs until it is killed.

Reproduced on master (dd44453f) and on v1.1.11.

Why

Migrate() short-circuits on the instant-DDL path: on success it calls
finalCleanup(), runs the OnSuccess hook and returns, so it never reads
<-mgtr.ghostTableMigrated.

https://github.com/github/gh-ost/blob/dd44453f489bdeb8bee5b93194a0a481e001e598/go/logic/migrator.go#L534-L552

Three goroutines then form a cycle:

  1. The changelog listener parks on the GhostTableMigrated send in
    onChangelogStateEvent. ghostTableMigrated is unbuffered and the
    instant-DDL path never reads it, so the send blocks. It is wrapped in
    base.SendWithContext, which only unblocks on context cancellation — and
    nothing cancels the context here, precisely because Migrate() has not
    returned. The listener is invoked from notifyListeners, which holds
    listenersMutex across the synchronous call, so the mutex stays held for as
    long as the send is parked.

  2. The binlog parser goroutine (go-mysql's BinlogSyncer.onStream) calls
    EventsStreamer.shouldDecodeRowsEvent through RowsEventDecodeFunc and
    blocks taking that same listenersMutex.

  3. BinlogSyncer.Close(), reached from finalCleanup() →
    EventsStreamer.Close(), ends in b.wg.Wait() and waits for that parser
    goroutine, which is never coming back.

Step 2 is what makes the cycle reachable. shouldDecodeRowsEvent was introduced
by #1699 ("Don't parse binlog events of unrelated tables", first released in
v1.1.11); it put listenersMutex on the binlog parser's goroutine, so a listener
parked while holding that mutex now stalls the parser directly, and with it
BinlogSyncer.Close().

--attempt-instant-ddl defaults to off and the short-circuit has no test
coverage, which is presumably why this has not been reported before. We hit it in
a downstream consumer that passes the flag for every migration, where every ALTER
MySQL could apply instantly hung.

Reproduction

Any instant-applicable ALTER on MySQL 8.0 with the flag set, e.g.

gh-ost --host=... --database=test --table=testing \
  --alter='ADD COLUMN author varchar(255)' \
  --attempt-instant-ddl --allow-on-master --execute

The column is added, Success! ... migrated instantly is logged, and the process
does not exit.

Fix

PR to follow: scope listenersMutex to the listener lookup in notifyListeners,
and give ghostTableMigrated a one-slot buffer.

Lenguaje dominante
Go
Estrellas
13.6k
Forks
1.4k
Merge medio
9 d 19 h
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de github/gh-ost

Todos los issues de github/gh-ost

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.