ClickHouse/ClickHouse

Clickhouse get lost when multiple CH instances are running on the same host

Open

#2,434 opened on May 28, 2018

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (8,400 forks)batch import
enhancementhelp wantedminor

Repository metrics

Stars
 (47,419 stars)
PR merge metrics
 (Avg merge 2d 2h) (1,000 merged PRs in 30d)

Description

On a machine with ip aliases, it is fairly easy to setup multiple clickhouse instances that will be binded to a specified ip.

with

inline bool isLocal(const Cluster::Address & address, const Poco::Net::SocketAddress & resolved_address, UInt16 clickhouse_port)
{
    ///    If there is replica, for which:
    /// - its port is the same that the server is listening;
    /// - its host is resolved to set of addresses, one of which is the same as one of addresses of network interfaces of the server machine*;
    /// then we must go to this shard without any inter-process communication.
    ///
    /// * - this criteria is somewhat approximate.
    ///
    /// Also, replica is considered non-local, if it has default database set
    ///  (only reason is to avoid query rewrite).

    return address.default_database.empty() && isLocalAddress(resolved_address, clickhouse_port);
}

Clickhouse will treat the two instances as in the same process, which is false, inducing errors in sharding.

I've seen the parameter treat_local_as_remote for the Cluster Class but that is not updateable and default to false

It would be nice to have a settable treat_local_as_remote like setting to enforce tcp instead of local process.

system.clusters table with 2 shards per host

SELECT *
FROM system.clusters

┌─cluster─────────────────────┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬──port─┬─is_local─┬─user────┬─default_database─┐
│ clickhouse2                 │         1 │            1 │           1 │ 10.0.0.1     │ 10.0.0.1     │  9000 │        1 │ default │                  │
│ clickhouse2                 │         2 │            1 │           1 │ 10.0.0.2     │ 10.0.0.2     │  9000 │        1 │ default │                  │
│ clickhouse2                 │         3 │            1 │           1 │ 10.0.0.3     │ 10.0.0.3     │  9000 │        0 │ default │                  │
│ clickhouse2                 │         4 │            1 │           1 │ 10.0.0.4     │ 10.0.0.4     │  9000 │        0 │ default │                  │
│ test_shard_localhost        │         1 │            1 │           1 │ localhost    │ 127.0.0.1    │  9000 │        0 │ default │                  │
│ test_shard_localhost_secure │         1 │            1 │           1 │ localhost    │ 127.0.0.1    │  9440 │        0 │ default │                  │
└─────────────────────────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴───────┴──────────┴─────────┴──────────────────┘

Changing the instance port is a working turn around which I'd like to avoid.

system.clusters table with 2 shards per host with custom tcp ports

<yandex>
   <tcp_port>9001</tcp_port>
</yandex>
SELECT *
FROM system.clusters

┌─cluster─────────────────────┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬──port─┬─is_local─┬─user────┬─default_database─┐
│ clickhouse2                 │         1 │            1 │           1 │ 10.0.0.1     │ 10.0.0.1     │  9001 │        1 │ default │                  │
│ clickhouse2                 │         2 │            1 │           1 │ 10.0.0.2     │ 10.0.0.2     │  9002 │        0 │ default │                  │
│ clickhouse2                 │         3 │            1 │           1 │ 10.0.0.3     │ 10.0.0.3     │  9001 │        0 │ default │                  │
│ clickhouse2                 │         4 │            1 │           1 │ 10.0.0.4     │ 10.0.0.4     │  9002 │        0 │ default │                  │
│ test_shard_localhost        │         1 │            1 │           1 │ localhost    │ 127.0.0.1    │  9000 │        0 │ default │                  │
│ test_shard_localhost_secure │         1 │            1 │           1 │ localhost    │ 127.0.0.1    │  9440 │        0 │ default │                  │
└─────────────────────────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴───────┴──────────┴─────────┴──────────────────┘

ch: 1.1.54378 Enterprise linux

Contributor guide