cube-js/cube

`UInt64` value returned from ClickHouse is truncated on Cube side

Open

#6,182 opened on Feb 21, 2023

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Rust (19,563 stars) (1,965 forks)batch import
driver:clickhousehelp wanted

Description

Describe the bug We have such dimension in Sales cube. Clickhouse is an underlying database

"saleKey":{
   "sql":"`farmHash64(doesnt matter what is inside)`",
   "type":"`number`",
   "primaryKey":true,
   "shown":true
},

farmHash64 returns UInt64 — [0 : 18446744073709551615]

Lets run script

select saleKey
from public.Sales limit 4

We are getting response

saleKey
8307928136669839400
12604678624050645000
15207895910732839000
16946569575044143000

But corresponding sql query returns other values. Attaching response from redis cache as an evidence


{
    "time": 1676967618149,
    "result": [
        {
            "sales__sale_key": "8307928136669839578"
        },
        {
            "sales__sale_key": "12604678624050644212"
        },
        {
            "sales__sale_key": "15207895910732839290"
        },
        {
            "sales__sale_key": "16946569575044142817"
        }
    ],
    "renewalKey": "SQL_QUERY_RESULT_STANDALONE_d1eff3196b1ea88672b632c472d243bc"
}

It easy to see that last 4 digits are not the same, why?

Expected behavior Values returned via script and sql-api are the same

Version: cubejs/cube:v0.30.62

Additional request Pls point to the source code line where truncation happens, if possible)

Contributor guide

Tech stack
rustsql
Domain
backenddatabase
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
ClickHouse UInt64 understandingCube schema basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
55
Research direction
Investigate Cube's ClickHouse driver for handling of UInt64 values. The issue may be in the conversion from UInt64 to JavaScript number (only 53 bits precision). Look at files in the 'cube backend' or 'cube client' Rust source for ClickHouse result serialization. The truncation likely occurs when constructing the JSON response from query results.