[Bug] createPartition / replacePartition on a non-master FE return no status, so auto-partition loads filter the rows instead of failing

Abierto Apto para principiantes
  • ¿Algo no está bien?
#70,258 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
82/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
cpp, java
Área
backend-api-design, databases

Línea de trabajo

Lee fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java en createPartition y replacePartition; después, revisa los tipos de resultado para confirmar cómo se devuelve el estado. Establece el estado de error en ambas ramas de no maestro y añade o actualiza pruebas específicas para esas respuestas. La tarea está completa cuando ambos resultados RPC indican NOT_MASTER al llamarlos en un seguidor.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Search before asking
  • I had searched in the issues and found no similar issues.
Version

4.1.4. master is the same: FrontendServiceImpl.createPartition (fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java:4732-4737) and FrontendServiceImpl.replacePartition (:5056-5061). Both have done this since the isMaster() check was added in #24412.

What's Wrong?

When createPartition or replacePartition is called on an FE that is not the master, the handler builds a NOT_MASTER status but returns the result without setting it:

TStatus errorStatus = new TStatus(TStatusCode.RUNTIME_ERROR);
if (!Env.getCurrentEnv().isMaster()) {
    errorStatus.setStatusCode(TStatusCode.NOT_MASTER);
    errorStatus.addToErrorMsgs(NOT_MASTER_ERR_MSG);
    LOG.warn("failed to createPartition: {}", NOT_MASTER_ERR_MSG);
    return result;   // result.status is never set
}

status is optional in TCreatePartitionResult / TReplacePartitionResult, so the BE receives the default TStatus, whose status_code is 0 (OK). In VRowDistribution::automatic_create_partition (be/src/exec/sink/vrow_distribution.cpp:148-159) the BE then takes the success path with an empty partition list. When the batched rows are replayed (VTabletWriter::_send_new_partition_batch, with _deal_batched = true), OlapTabletFinder::find_tablets is called without a missing-rows list. Rows that still have no partition are therefore counted as filtered ("no partition for this tuple").

The result depends on the load's filter tolerance:

  • With max_filter_ratio > 0 (or strict_mode off for INSERT), the load can succeed while silently dropping every row that needed a new partition.
  • With no tolerance, it fails with a filtered-rows error that hides the real cause, NOT_MASTER.

The BE sends these RPCs to the master address it learned from heartbeats. So this happens when that address is briefly stale, for example during an FE master switch, or when the old master restarts as a follower. replacePartition (INSERT OVERWRITE with auto-detect partitions) has the same missing setStatus; we have not traced what that path then does with the rows.

What You Expected?

A non-master FE should return the NOT_MASTER status, so the BE fails the request (or retries against the new master) instead of treating it as a successful call that created nothing.

How to Reproduce?

We found this while reading the code and have not reproduced it by forcing a failover. Two ways to see it:

  1. Call FrontendService.createPartition over thrift on a follower FE. The returned TCreatePartitionResult has no status field set.
  2. On an auto-partition table, run a Stream Load with max_filter_ratio=1 whose rows need a new partition, and switch the FE master while the load runs (stop the master FE so a follower takes over before the BEs' next heartbeat). The load can finish with those rows counted as filtered.
Anything Else?

The fix is one line in each handler: result.setStatus(errorStatus); before return result; in the !isMaster() branch. Separately, the BE could treat a result with no status set as an error rather than reading the default OK, which would cover any other handler with the same mistake.

Are you willing to submit PR?
  • Yes I am willing to submit a PR!
Code of Conduct
Lenguaje dominante
Java
Estrellas
16.2k
Forks
4k
Merge medio
1 d 17 h
PR fusionados (30 d)
660

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/doris

Todos los issues de apache/doris

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.