Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Bug] BanyanDB trace block metadata format changed in 0.11 without a file format version bump

Abierto
  • ¿Algo no está bien?
#14,169 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@hanahmily ya está trabajando en esto.

Desde el 26/10/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

bug database
Search before asking
  • I had searched in the issues and found no similar issues.
Apache SkyWalking Component

BanyanDB (apache/skywalking-banyandb)

What happened

apache/skywalking-banyandb#1237 changed the on-disk encoding of trace blockMetadata. It inserts a timestamp-bounds marker and the min/max timestamps between count and spans (marshal, unmarshal, skip). The readers accept only the new layout.

The file format version was not bumped. It is still 1.5.0, the same as in 0.10.3, and versions.yml still lists 1.4.0 and 1.5.0 (also on main, now in pkg/fileformat). Earlier on-disk format changes did bump it, for example apache/skywalking-banyandb#665 (1.2.0 → 1.3.0), apache/skywalking-banyandb#925 (1.3.0 → 1.4.0) and apache/skywalking-banyandb#1051 (1.4.0 → 1.5.0).

Because of this, the compatibility check described in docs/operation/upgrade.md passes when 0.11 opens trace data written by 0.10. The server starts normally, and the incompatibility only appears at query time. Every trace query whose time range touches a 0.10 part fails as a whole:

execute the query plan for trace segment: cannot iterate tstIter: cannot read primary block for part
"Oct 25 00:00:00, Oct 25 04:33:26, 21 MB, 67 MB, 17,008, 11,784" at offset 73553 with size 24852:
cannot unmarshal index block: cannot skip blockMetadata entry: cannot skip timestamp bounds marker: 4447272

The reader takes the old spans.offset for the new marker and rejects any value greater than 1.

In the same upgrade, measure, stream and property data written by 0.10.3 were read correctly by 0.11.1. OAP 11.0.0 returned the same results as OAP 10.4.0 on 0.10.3. Only trace data is affected. Neither the 0.11.0 CHANGES nor the "Upgrading to 0.11" section of the upgrade guide mentions a trace data format change.

What you expected to happen

A change to the on-disk format should come with a file format version bump, as earlier releases did. The incompatibility is then detected and handled explicitly, instead of every trace query failing after a successful start.

Ideally, 0.11 should also keep reading trace data written by 0.10. Otherwise users who upgrade have to drop up to a full TTL of traces. The new layout already supports blocks whose timestamp bounds are unknown, so old blocks could be read that way.

Data that 0.11.0 and 0.11.1 have already written also carries version 1.5.0 but uses the new layout. A bump in the next release therefore cannot rely on 1.5.0 alone to identify the old layout.

How to reproduce
  1. Run BanyanDB 0.10.3 in standalone mode with OAP 10.4.0, and let an agent report some traces.
  2. Stop OAP and BanyanDB. On the same data directory, start BanyanDB 0.11.1 and OAP 11.0.0.
  3. BanyanDB starts without any version error.
  4. Query traces over a time range that includes the data written by 0.10.3, for example queryTraces with queryOrder: BY_DURATION. The query fails with the error above.
Anything else

Environment: standalone mode, arm64 (AWS t4g), Docker images apache/skywalking-banyandb:0.10.3 → 0.11.1 and apache/skywalking-oap-server:10.4.0 → 11.0.0. Schema registry is property-based (no etcd). Segment metadata on disk before the upgrade is {"version":"1.5.0","endTime":...}.

I'm glad to help with a fix once the preferred direction is clear.

Are you willing to submit a pull request to fix on your own?
  • Yes I am willing to submit a pull request on my own!
Code of Conduct
  • I agree to follow this project's Code of Conduct
Lenguaje dominante
Java
Estrellas
25k
Forks
6.6k
Merge medio
15 h 37 min
PR fusionados (30 d)
32

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/skywalking

Todos los issues de apache/skywalking

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.