Fixed-length strings read past the end of their field

Open
  • Is something wrong?
#504 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@mattjala is already working on this.

Since Oct 13, 2026.

Assessment

This issue has not been assessed yet.

Description

Component - Object Library

A fixed-length string only carries a terminator when its field is longer than the text it
holds. The HDF5 JNI reads these strings by handing the raw element address to
NewStringUTF, which reads until it finds a terminator, so a string that fills its field
runs on into whatever follows it. HDFView displays those neighbouring bytes as part of the
value wherever it reads the string through H5DreadVL.

The data is already wrong when it crosses the JNI boundary, so HDFView cannot fix the
values. It can detect the condition, which is covered at the end.

Root cause

java/src-jni/jni/h5util.c in the HDF5 source, the H5T_STRING case of
translate_atomic_rbuf:

else {
    if (NULL == (jobj = ENVPTR->NewStringUTF(ENVONLY, char_buf))) {

char_buf points at the element. Nothing bounds the read to the element's size. The
variable-length branch just above it is fine, because those strings are always terminated.

Worth noting that H5Dread_string in the same source tree already handles this correctly
for the same kind of field, by copying out the element's own bytes and terminating the
copy itself.

For the file PacketTableIssue.h5 at /rec/Events, where each member is a variable-length
sequence of one byte strings. h5dump shows ("I","n","i","t",...):

before:  [Initialize, nitialize, itialize, tialize, ialize, alize, lize, ize, ze]
after:   [I, n, i, t, i, a, l, i, z]

It also crosses member boundaries. A compound of {tag: 2 byte string, id: int, note: 4 byte string} with tag holding exactly ab and id given printable bytes:

before:  [abAAAAxy, 1094795585, xy]
after:   [ab, 1094795585, xy]

The two byte tag swallowed the int member and the string member after it. note is
unaffected because it is shorter than its field, so its padding still terminates it.

Origin

This is not a regression in HDFView. The unbounded read is present unchanged at the hdf5_2.0.0 release tag, which is what HDFView 3.4.1 shipped against, and on current develop.

Likely fix in HDF5

Copy at most the element's own bytes into a buffer, terminate the copy, and convert that,
which is what H5Dread_string already does.

Note that a Java side workaround would only correct the displayed value. The out of bounds
read itself happens in native code and can only be fixed there.

Affected HDFView test files

The defect only reaches HDFView through H5DreadVL, which in practice means a fixed-length
string sitting inside a variable-length type.

A fixed-length string that holds no
variable-length data is read with plain H5Dread into a byte array and converted by
Dataset.byteToString, which takes exactly the field's bytes with
new String(bytes, i * length, length) and trims at the first terminator. That is bounded,
so the library's unbounded read is never reached.

object/src/test/java/object/TestNestedDatatypeShapes.testVlenOfFixedString is already disabled and waiting for a JNI fix.

testArrayOfFixedString beside it is unaffected, because an array of fixed-length strings
contains no variable-length data and takes the flat path.

Next Steps for HDFView

The condition is detectable from the datatype, so HDFView can avoid rendering potentially incorrect values until a fix gets into the JNI itself. Once a JNI patch gets in, this can be removed.

Dominant language
Java
Stars
157
Forks
28
Avg merge
16h 22m
Merged PRs (30d)
12

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from HDFGroup/hdfview

All issues in HDFGroup/hdfview

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.