[client-v2] RowBinaryFormatWriter.setInputStream / setReader write the text "…$InputStreamHolder@…" instead of the stream data

Start with client-v2/src/main/java/com/clickhouse/client/api/data_formats/RowBinaryFormatWriter.java, focusing on setInputStream, setReader, and commitRow; then read SerializerUtils string and fixed-string serialization. Add regression coverage for the four overloads, lengths, and null inputs, including jdbc-v2 LOB setters if practical. Done when stream and reader contents are written correctly and existing setString, byte-array, and null behavior remains unchanged.

Description

RowBinaryFormatWriter.setInputStream(...) and setReader(...) (public methods of ClickHouseBinaryFormatWriter) do not write the data of the stream. The writer never reads the stream. It writes the toString() of a private wrapper object instead.

• String, Nullable(String), LowCardinality(String), String DEFAULT 'x': the column gets the text com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@43195e57 (or …$ReaderHolder@…). There is no error. This is silent data corruption.
• FixedString(N): IllegalArgumentException: length of byte array value is 87, which should NOT longer than N. The message does not tell the user why.
• Other types (Int32, Array(UInt8)): an error that names the wrapper class.
• A null stream or reader also writes the wrapper text. setValue(i, null) correctly writes NULL or the column default.

All four overloads are affected: setInputStream(int|String, InputStream, long) and setReader(int|String, Reader, long), with len = 3 and len = -1. After the insert, the ByteArrayInputStream still has all 3 bytes available.

jdbc-v2 reaches this code when beta.row_binary_for_simple_insert=true (WriterStatementImpl). All 10 LOB setters write the wrapper text: setBlob (3 overloads), setClob (3), setNClob (3), setSQLXML. When the writer is off, the InputStream / Reader overloads store the correct value.

Related: #3334 (jdbc-v2 writer setters with an empty body). Its suggested fix tells the fixer not to delegate to setInputStream / setReader until this issue is fixed.

Found by automated analysis of the client while working on #3334. Verified against a live server (not by inspection only).

Steps to reproduce

1. CREATE TABLE t_rbw_holder (id Int32, v String) ENGINE = MergeTree ORDER BY id
2. Insert one row with RowBinaryFormatWriter: setInteger(1, 1), setInputStream(2, new ByteArrayInputStream("abc".getBytes()), 3), commitRow() (see the code example).
3. SELECT v FROM t_rbw_holder

Error Log or Exception StackTrace

Direct client-v2 (one new table per row, RowBinaryWithDefaults, value "abc"):

== v String
  setInputStream(int,len=3)    -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@43195e57] | stream bytes left=3
  setInputStream(int,len=-1)   -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@223d2c72] | stream bytes left=3
  setInputStream(name,len=3)   -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@8f4ea7c] | stream bytes left=3
  setReader(int,len=3)         -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$ReaderHolder@10e31a9a] | stream bytes left=3
  setReader(int,len=-1)        -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$ReaderHolder@131774fe] | stream bytes left=3
  setReader(name,len=3)        -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$ReaderHolder@158d2680] | stream bytes left=3
  CONTROL setString(int)       -> stored [abc]
== v FixedString(3)
  setInputStream(int,len=3)    -> ERROR IllegalArgumentException: length of byte array value is 87, which should NOT longer than 3
  setReader(int,len=3)         -> ERROR IllegalArgumentException: length of byte array value is 81, which should NOT longer than 3
  CONTROL setString(int)       -> stored [abc]
== v Int32
  setInputStream(int,len=3)    -> ERROR IllegalArgumentException: Cannot convert com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@18078bef to Integer
== v Nullable(String)
  setInputStream(2,null,-1)    -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@56620197] (v IS NULL = 0)
  setReader(2,null,-1)         -> stored [com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$ReaderHolder@36bc55de] (v IS NULL = 0)
  CONTROL setValue(2,null)     -> stored [NULL] (v IS NULL = 1)

Nullable(String), LowCardinality(String) and String DEFAULT 'x' give the same result as String for all six overloads. FixedString(64) gives the same error as FixedString(3).

jdbc-v2, INSERT INTO t VALUES (?, ?), v Nullable(String), value "abc":

Note: when the writer is off, the Blob / Clob / NClob / SQLXML object overloads fail in the SQL-text path (PreparedStatementImpl.encodeObject renders x.toString() without quotes, server Code: 62). That is a different defect, and this issue does not include it.

Expected Behaviour

The column gets the data of the stream or reader. setInputStream(2, new ByteArrayInputStream("abc".getBytes()), 3) into String stores abc, the same as setString(2, "abc") and setValue(2, "abc".getBytes()). A null stream or reader writes NULL (or the column default), the same as setValue(2, null). jdbc-v2 with the writer on stores the same value as with the writer off.

Root cause

client-v2/src/main/java/com/clickhouse/client/api/data_formats/RowBinaryFormatWriter.java (main f9586aba0):

• setInputStream / setReader (:257-:274) store new InputStreamHolder(in, len) / new ReaderHolder(reader, len) (private classes, :276-:292) with setValue. No code in the repository reads these classes.
• commitRow() (:97-:98) gives the holder to RowBinaryFormatSerializer.writeValuePreamble (the holder is not null, so it writes the "not null" marker, also for a null stream) and then to SerializerUtils.serializeData.
• SerializerUtils.serializePrimitiveData, case String (:711-:718): the value is not byte[] or StringValue, so it calls convertToString(value) (:1084), which is String.valueOf(value), which is Object.toString(). case FixedString (:720-:727) writes the same text and fails the length check.

The holders were added with the writer in b2fe6bbaa ("Implemented RowBinaryWriter for simple inserts under beta flag").

jdbc-v2 callers: WriterStatementImpl.setClob (:316, :322, :328), setBlob (:334, :340, :346), setNClob (:352, :358, :364), setSQLXML (:370).

Suggested fix

In setInputStream / setReader, read the data when the method is called and store a normal value:

• InputStream → byte[] (read len bytes when len >= 0, else read to the end), then setValue(i, bytes). The String and FixedString serializers already accept byte[] (SerializerUtils.java:712, :721).
• Reader → String (same length rule), then setValue(i, string).
• null stream or reader → setValue(i, null).
• Remove InputStreamHolder / ReaderHolder.

Another option is to unwrap the holders in SerializerUtils.serializeData. But the String encoding writes the length before the bytes, so a stream with len = -1 must be read into memory first in both options. If the stream has fewer than len bytes, give an error, do not write a short value.

Behavior that must not change: setString, setValue(i, byte[]) and setValue(i, null). A regression test can insert through each of the four overloads into String, Nullable(String), FixedString(N) and String DEFAULT 'x' (with len = exact length and -1, and with a null stream), then read the value back. For jdbc-v2, the same test can compare each LOB setter with the writer on and off.

Code Example

import com.clickhouse.client.api.Client;
import com.clickhouse.client.api.data_formats.RowBinaryFormatWriter;
import com.clickhouse.client.api.insert.InsertSettings;
import com.clickhouse.client.api.metadata.TableSchema;
import com.clickhouse.data.ClickHouseFormat;
import java.io.ByteArrayInputStream;

public class Minimal {
    public static void main(String[] args) throws Exception {
        try (Client client = new Client.Builder()
                .addEndpoint("http://localhost:8123")
                .setUsername("default").setPassword("<password>")
                .serverSetting("async_insert", "0")
                .build()) {
            client.execute("DROP TABLE IF EXISTS t_rbw_holder").get().close();
            client.execute("CREATE TABLE t_rbw_holder (id Int32, v String) ENGINE = MergeTree ORDER BY id").get().close();
            TableSchema schema = client.getTableSchema("t_rbw_holder");
            ClickHouseFormat format = ClickHouseFormat.RowBinaryWithDefaults;
            ByteArrayInputStream in = new ByteArrayInputStream("abc".getBytes());
            client.insert("t_rbw_holder", out -> {
                RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);
                w.setInteger(1, 1);
                w.setInputStream(2, in, 3);
                w.commitRow();
            }, format, new InsertSettings()).get().close();
            client.queryAll("SELECT v FROM t_rbw_holder")
                    .forEach(r -> System.out.println(r.getString("v")));
            // prints: com.clickhouse.client.api.data_formats.RowBinaryFormatWriter$InputStreamHolder@...
            // expected: abc
            System.out.println("bytes left in stream: " + in.available()); // 3
        }
    }
}

Configuration

Client Configuration

// client-v2: defaults, RowBinaryWithDefaults (also seen with the jdbc-v2 writer on)
Properties p = new Properties();
p.setProperty("beta.row_binary_for_simple_insert", "true"); // jdbc-v2 path only

Environment

• Cloud
• Client version: client-v2 / jdbc-v2 0.12.0-rc1-SNAPSHOT, built from main f9586aba0
• Language version: Java 17 (Temurin 17.0.21)
• OS: Linux (Docker)

ClickHouse Server

• ClickHouse Server version: 26.9.1.1629
• ClickHouse Server non-default settings, if any: none
• CREATE TABLE statements for tables involved: CREATE TABLE t_rbw_holder (id Int32, v String) ENGINE = MergeTree ORDER BY id (also Nullable(String), LowCardinality(String), String DEFAULT 'x', FixedString(3), FixedString(64), Int32, Array(UInt8) for v)
• Sample data for all these tables: not needed, the example inserts the row