[Feature] Add a DECIMAL (BigDecimal) property data type for exact amounts

Open
#3,206 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Stale
Tech stack
java
Domain
api, database

Research direction

Start by reviewing the implementation on the feat/decimal-datatype branch and the proposed changes across DataType, BytesBuffer, PropertyKey, BatchAPI, and the struct copy. Run DataTypeTest, BytesBufferTest, JsonUtilTest, VertexCoreTest, and VertexApiTest with the listed CI scripts; done means the four encoding/API points are resolved and all affected unit, core, API, and struct tests pass.

Written by the indexing model from the issue text.

Description

Feature Description (功能描述)

Add an arbitrary-precision decimal property type: DataType.DECIMAL backed by java.math.BigDecimal, with REST data_type: DECIMAL and PropertyKey.Builder.asDecimal().

Motivation

The numeric property types today are BYTE/INT/LONG/FLOAT/DOUBLE. Values that do not fit a long and must not be rounded (token balances in wei go up to 2^256 - 1, 78 digits; money amounts in general) can only be stored as TEXT. That loses the one place where the server itself does arithmetic: update_strategies in PUT /graph/{vertices,edges}/batch (SUM / BIGGER / SMALLER). UpdateStrategy already computes in BigDecimal, but the result goes back to the property's type, so with DOUBLE a SUM of 10^18 + 1 is 10^18; TEXT fails the strategy's Number type check. For accumulating balances during an import this blocks the use case.

Proposal (points to confirm)
  1. Type code 12 in DataType (server and the hugegraph-struct copy), name "decimal". Please confirm the code is free.
  2. No sort key, no index, no OLAP range. isNumber() stays false; the schema builders reject these with an explicit message. Reason: there is no fixed-width byte-order-preserving encoding for a decimal, and faking one through LongEncoding would be lossy. SUM/MAX/MIN aggregate types on the property key are allowed as for numbers.
  3. Encoding in BytesBuffer: vint(len) + unscaled two's-complement bytes + vint(scale). Exact for any precision, 33 bytes for a uint256, scale preserved; existing encodings untouched.
  4. JSON: always a string on output (toPlainString()), a string or a number literal accepted on input. A JSON number is a double to most clients; a string is the only lossless representation. In the batch update a fraction has to be sent as a string, because the request's properties map is parsed by Jackson before any schema is known (a 0.000000000000000001 literal becomes a double); integral literals are exact.

Also in the same change: exact comparison in ConditionQuery when one side is a BigDecimal (instead of through doubleValue()), a string variant in the store-side row decoder, and one fix in BatchAPI.updateExistElement: the JSON value is normalised through the property key before the strategy runs, on both paths. Today the strategy receives the raw JSON value, which only works for the types Jackson happens to produce; a decimal (or a date) sent as a string fails the type check.

Status

Implementation and tests are ready on the branch feat/decimal-datatype in my fork (one commit on top of master 60c8803, 25 files): unit tests (DataTypeTest, BytesBufferTest, JsonUtilTest), core (PropertyKey/IndexLabel/EdgeLabel/VertexCoreTest), API (VertexApiTest: SUM on 2^256-2 + 1, two entries of one vertex within one request, BIGGER) and struct. Run locally with the CI scripts: unit, core on rocksdb and memory, api on rocksdb, all green. I will open the PR tomorrow to leave time for comments on the points above; the client/loader in hugegraph-toolchain will be a separate change.

If any of the four points looks wrong, I would rather hear it here than in the code review.

Dominant language
Java
Stars
3.2k
Forks
637
Avg merge
3d 18h
Merged PRs (30d)
23

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/hugegraph

All issues in apache/hugegraph

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.