[client-v2] insert() always sends the INSERT statement in the URL, so wide column lists fail with "HTML Form Exception: Field value too long" (http_max_field_value_size, 128 KiB)

Start with Client.java:1855 and HttpAPIClientHelper.java:560,881 to trace how the insert statement reaches the URL and how the streaming request is built. Compare that path with the body-based request at HttpAPIClientHelper.java:640, then run the supplied UrlLimitReproTest reproduction. Done means the 700-column insert succeeds, or the client reports a clear error naming the limit.

Description

Client.insert(...) builds the INSERT INTO <table> (<columns>) FORMAT <format> statement and attaches it as the URL query-string parameter query, because the request body is reserved for the data stream:

• client-v2/src/main/java/com/clickhouse/client/api/Client.java:1855 — requestSettings.serverSetting(ClickHouseHttpProto.QPARAM_QUERY_STMT, sqlStmt.toString());
• client-v2/src/main/java/com/clickhouse/client/api/internal/HttpAPIClientHelper.java:560 (createRequestURI) → addStatementParams(...) puts it on the URIBuilder. The insert path (createRequest(server, requestConfig, IOCallback<OutputStream>), line 881) always passes isMultipartRequest = false, so there is no body placement for the statement.

The server rejects any single URL query-string value longer than http_max_field_value_size (131072 bytes by default, measured on the URL-encoded text) in HTMLForm::readQuery (src/Server/HTTP/HTMLForm.cpp), before max_query_size or http_max_uri_size (1 MiB) come into play. The limit cannot be raised with a query-level setting, because the server parses the URL query string with its default settings.

So an insert into a table with a wide column list (many columns and/or long column names) fails with an opaque Poco error, even though the same statement executes fine through query()/queryAll() (which sends SQL in the body). There is currently no client-side option to move the insert statement out of the URL, and the limit is not documented.

This is the Java counterpart of ClickHouse/clickhouse-cs#702 (there the statement placement is configurable via InsertQueryPlacement.Body, and the issue is that the docs name http_max_uri_size instead of http_max_field_value_size). In client-v2 there is no equivalent switch, so the limit is a hard failure rather than a documentation inaccuracy.

ClickHouse server version

26.10.2.103 (local, default settings), via SELECT version().

Reproduction

client-v2/src/test/java/com/clickhouse/client/UrlLimitReproTest.java:

package com.clickhouse.client;

import com.clickhouse.client.api.Client;
import com.clickhouse.client.api.DataStreamWriter;
import com.clickhouse.client.api.insert.InsertSettings;
import com.clickhouse.data.ClickHouseFormat;
import org.testng.annotations.Test;

import java.io.OutputStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class UrlLimitReproTest {

    @Test(groups = {"unit"})
    public void testLongInsertStatementInUrl() throws Exception {
        try (Client client = new Client.Builder()
                .addEndpoint("http://localhost:8123")
                .setUsername("default").setPassword("")
                .setDefaultDatabase("default")
                .compressClientRequest(false)
                .build()) {
            System.out.println("server version: " + client.queryAll("SELECT version()").get(0).getString(1));
            run(client, 500);
            run(client, 700);
        }
    }

    private static void run(Client client, int cols) {
        String table = "repro_url_limit_" + cols;
        List<String> names = new ArrayList<>();
        for (int i = 0; i < cols; i++) {
            names.add(String.format("c%04d_", i) + repeat("x", 194));
        }
        try {
            client.queryAll("DROP TABLE IF EXISTS " + table);
            // note: this CREATE TABLE is >140k chars for cols=700 and succeeds, because query() sends SQL in the body
            client.queryAll("CREATE TABLE " + table + " ("
                    + names.stream().map(n -> n + " UInt8").collect(Collectors.joining(", "))
                    + ") ENGINE = Memory");

            String stmt = "INSERT INTO " + table + " (" + String.join(", ", names) + ") FORMAT RowBinary";
            System.out.println("cols=" + cols + " statement chars=" + stmt.length());

            DataStreamWriter writer = new DataStreamWriter() {
                @Override
                public void onOutput(OutputStream out) throws java.io.IOException {
                    byte[] row = new byte[cols];
                    Arrays.fill(row, (byte) 1);
                    out.write(row);
                    out.flush();
                }
            };
            client.insert(table, names, writer, ClickHouseFormat.RowBinary, new InsertSettings()).get();
            System.out.println("cols=" + cols + " -> OK, count()="
                    + client.queryAll("SELECT count() FROM " + table).get(0).getLong(1));
        } catch (Exception e) {
            Throwable t = e;
            while (t.getCause() != null) t = t.getCause();
            System.out.println("cols=" + cols + " -> FAILED: " + t.getClass().getName() + ": "
                    + String.valueOf(t.getMessage()).split("\n")[0]);
        }
    }

    private static String repeat(String s, int n) {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < n; i++) sb.append(s);
        return sb.toString();
    }
}

Run:

mvn -f client-v2/pom.xml test -Dtest=UrlLimitReproTest

Actual output (target/surefire-reports/com.clickhouse.client.UrlLimitReproTest-output.txt), on main at 74b0a62-era (client-v2 0.12.0-rc1-SNAPSHOT):

server version: 26.10.2.103
cols=500 statement chars=101049
cols=500 -> OK, count()=1
cols=700 statement chars=141449
cols=700 -> FAILED: com.clickhouse.client.api.ServerException: Code: 1000, e.code() = 0, HTML Form Exception: Field value too long  (queryId= )

Expected: both inserts succeed (as the equally long CREATE TABLE does through query()), or at minimum the client fails with a clear, documented client-side error naming the limit instead of surfacing a raw Poco form-parsing exception.

Suggested fix

Options, in rough order of usefulness:

1. Send the insert statement in the request body instead of the URL when it is large — the server already accepts a multipart/form-data body with a query field, which client-v2 uses for statement parameters (HttpAPIClientHelper.createRequest(..., String body), line 640, behind HTTP_SEND_PARAMS_IN_BODY). Extending the same mechanism to the insert path (createRequest(..., IOCallback<OutputStream>), line 881) would remove the limit.
2. If that is not desired, add a client/insert-level option for statement placement (as clickhouse-cs has), and document that the statement goes in the URL and is therefore limited to http_max_field_value_size (128 KiB by default, on the URL-encoded text) — not http_max_uri_size (1 MiB), and not raisable with a query-level setting.

Link

Related: https://github.com/ClickHouse/clickhouse-cs/issues/702

Found by automated cross-client analysis; verified against a live ClickHouse 26.10 server with the test above.