Compound members vanish when a variable-length string member is NULL

Open
  • Is something wrong?
#503 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@mattjala is already working on this.

Since Oct 13, 2026.

Assessment

This issue has not been assessed yet.

Description

Component - Object Library

When a compound has a variable-length string member whose stored value is NULL, that
member does not come back from the HDF5 JNI at all. The member list is one entry short,
so every member after the NULL moves up a place and HDFView reads the wrong member for
each of them. Reading the last position throws IndexOutOfBoundsException, which is what
surfaces in the table as *ERROR*.

The data is already wrong when it crosses the JNI boundary, before HDFView sees it, so
HDFView cannot fix the values. It can detect the condition, which is covered at the end.

Root Cause

java/src-jni/jni/h5util.c in the HDF5 source. translate_rbuf appends each compound
member's converted value to the row only when the conversion produced an object:

jobj = translate_atomic_rbuf(ENVONLY, memb, memb_vlClass,
                             char_buf + i * typeSize + memb_offset, memb_vlSize);
if (jobj) {
    ENVPTR->CallBooleanMethod(ENVONLY, jList, arrAddMethod, (jobject)jobj);
    ...
}

A NULL variable-length string deliberately converts to a Java null a few lines earlier,
to avoid passing NULL to NewStringUTF, so the member is skipped instead of recorded.

The same guard appears in three places:

  • the compound member loop in translate_rbuf, for a top level compound
  • the compound member loop in translate_atomic_rbuf, for a compound nested in another type
  • the element loop for a sequence of atomic values, so an array of strings also loses its
    NULL entries and comes back short
Reproducing

This can be reproduced with a compound of {int id, varstr name, double val} with nothing written to it. The default
fill is zero bytes, so each string pointer reads back as NULL:

nmembers in datatype = 3
row 0 size=2 contents=[0, 0.0]
row 1 size=2 contents=[0, 0.0]

Three members in the datatype, two values in the row, and val has moved into name's
position.

On a real file, bug0951_StitchTest19_4_25_raw1.h5 at
/Groups/StichTest/StitchTest/0x00000000/Applications/CCD-Acquire/MeshFitPara, where
h5dump shows one KeyList element as
("RetardationRatio"), DOUBLE, NULL, 0.2, 0, 1000:

before:  list size=5   [RetardationRatio], 1, 0.2, 0.0, 1000
after:   list size=6   [RetardationRatio], 1, null, 0.2, 0.0, 1000
Origin

This is not a regression in recent HDFView. The guard dates to HDF5 #2156 and #2232 in late 2022 and is present
unchanged at the hdf5_2.0.0 release tag, which is what HDFView 3.4.1 shipped against,
as well as on current develop. It became visible in the 3.5.0 pre-release because the nested compound
work made these datasets readable at all. In 3.4.1 the same files could not be opened.

The FFM implementation of the Java bindings, which is the default on Java 25 and above,
does not have this problem. It keeps the member's position and returns an empty string.
Only the JNI implementation is affected.

Likely fix in HDF5

Drop the guard at all three sites and store the value whether or not it is null. An
ArrayList accepts null, and DeleteLocalRef on NULL is a no op, so this is a small change.

Next Steps for HDFView

The condition is detectable from the datatype, so HDFView can avoid rendering potentially incorrect values until a fix gets into the JNI itself. Once a JNI patch gets in, this can be removed.

Dominant language
Java
Stars
157
Forks
28
Avg merge
16h 22m
Merged PRs (30d)
12

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from HDFGroup/hdfview

All issues in HDFGroup/hdfview

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.