Model time features drop sub-second precision

Open
#148 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
Half a day
Newbie friendliness
55/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python

Research direction

The floor division by NANOSECONDS_IN_SECONDS is at packages/kumo-core/src/kumo_core/encoders/tensor_frame_encoder.py lines 213 and 223, and packages/kumo-core/src/kumo_core/encoders/tensor_mapper.py line 777 does the same with 10**9. Start by checking how timestamp_unit reaches these encoders, then keep the sub-second remainder. Done means two events 300 ms apart get different time features on a millisecond index, and seconds-index encoder outputs stay bit-identical.

Written by the indexing model from the issue text.

Description

Problem

On an index with a sub-second clock (graph.snapshot(timestamp_unit="milliseconds"), #123), two events 300 ms apart are visible, ordered and windowed correctly, but the model still sees them at the same moment.

Cause

The encoders turn every timestamp into whole seconds before building features:

  • packages/kumo-core/src/kumo_core/encoders/tensor_frame_encoder.py:213 seed_time // NANOSECONDS_IN_SECONDS
  • packages/kumo-core/src/kumo_core/encoders/tensor_frame_encoder.py:223 time.view(-1, 1) // NANOSECONDS_IN_SECONDS
  • packages/kumo-core/src/kumo_core/encoders/tensor_mapper.py:777 unix // 10**9

So relative-time and timestamp features are identical for events within the same second.

Proposed fix

Keep the sub-second part when computing time features, for example by dividing as floats or carrying the remainder. On a seconds index every timestamp is a whole second, so the model's inputs stay exactly as today.

Acceptance

  • On a millisecond index, two events 300 ms apart get different time features.
  • On a seconds index, encoder outputs are bit-identical to today's.
Dominant language
Python
Stars
20
Forks
6
Avg merge
1d 21h
Merged PRs (30d)
41

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/kumo-engine

All issues in NVIDIA/kumo-engine

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.