Standardize temporal column statistics across pandas, Polars, and the UI

Start by comparing the pandas and Polars statistics modules and their listed tests, then inspect DAGWorksDescribe.tsx and the observability schema or generated types. Run the focused SDK tests and existing frontend tests to establish current payload and rendering behavior. Done means the documented temporal contract is implemented across both producers, schemas, renderers, and cross-backend/frontend tests, including timezone and malformed-value cases.

Problem

Hamilton's observability schema currently treats several distinct temporal concepts as if they were JavaScript-compatible datetimes. pandas and Polars consequently produce different payloads for equivalent columns, and some payloads either cannot be serialized or render incorrectly in the UI.

This was identified while reviewing #1675 and Jernej's comment in https://github.com/apache/hamilton/pull/1675#discussion_r3795651892. PR #1675 should remain a focused fix: Polars Time and Duration are routed to the existing unhandled path so they no longer break datetime rendering. This issue tracks the complete cross-backend solution.

Current behavior

Logical type	pandas	Polars	UI consequence
Datetime	base_data_type=\"datetime\"; ISO-like statistics	base_data_type=\"datetime\"; ISO-like statistics	Existing datetime renderer generally works
Date	Python date object columns normally become unhandled	base_data_type=\"datetime\"; min/max are YYYY-MM-DD	new Date(\"YYYY-MM-DD\") is interpreted as UTC and can display the previous day in negative-offset timezones
Time	Object columns normally become unhandled	unhandled after #1675	No rich statistics, but safe
Duration/timedelta	base_data_type=\"datetime\"; min/max/mean/quantiles contain raw pd.Timedelta	unhandled after #1675	pandas payload fails ordinary JSON serialization; duration strings are not valid JavaScript Dates anyway

The frontend datetime path in ui/frontend/src/components/dashboard/Runs/Task/result-summaries/DAGWorksDescribe.tsx calls new Date(...) for range, quantile, and histogram values. That operation does not preserve Date semantics and cannot parse Time or Duration values.

The generated frontend datetime type and backend Pydantic datetime model also do not declare all fields that producers currently emit, which prevents static typing from catching contract drift.

Desired solution

Define one explicit temporal statistics contract shared by pandas, Polars, the backend schema, generated frontend types, and UI renderers.

Logical types

Use distinct base_data_type values for:

1. datetime
2. date
3. time
4. duration

Do not route Date, Time, or Duration through the datetime renderer.

Canonical JSON representations

The precise representation should be documented and applied identically by both producers. A suggested starting point:

• Datetime: ISO 8601 datetime strings with an explicit, documented timezone policy. Preserve offsets for timezone-aware values.
• Date: YYYY-MM-DD strings. Treat them as calendar dates in the UI; do not pass them directly to new Date(string).
• Time: ISO local-time strings such as HH:mm:ss[.ffffff], with an explicit policy for timezone-aware time values if supported.
• Duration: a numeric canonical unit, preferably integer microseconds, for min/max/mean/quantiles and charting. Avoid display-formatted strings as the transport representation. If ISO 8601 durations are selected instead, the UI must parse them explicitly and tests must cover ordering/charting.

All emitted statistics—including min, max, mean, standard deviation, quantiles, and histogram boundaries—must be JSON-safe and use the canonical representation for their logical type.

A schema-version bump may be required if adding base_data_type values is not compatible with the current dagworks_describe v0.0.3 contract.

UI behavior

Add explicit dispatch and rendering for each type:

• Datetime: retain the existing renderer after confirming timezone behavior.
• Date: render calendar dates without UTC conversion or timezone shifting. Construct from components or display the canonical value through a date-aware formatter.
• Time: render wall-clock values without inventing a date or applying a browser timezone.
• Duration: format canonical numeric values into readable durations and chart them numerically.

Unsupported or malformed values should degrade to the existing unhandled presentation rather than producing Invalid Date or NaN.

Schema and generated types

Update the backend observability models and generated frontend API types so each temporal statistics shape declares the fields actually emitted and consumed. Avoid casting datetime records to numeric statistics types in the frontend.

Acceptance criteria

• Equivalent pandas and Polars columns produce the same logical base_data_type and value shape for Datetime, Date, Time, and Duration.
• Every pandas and Polars temporal payload passes JSON serialization without custom fallback encoders.
• Datetime timezone/offset behavior is documented and tested.
• Date values display the same calendar day in positive and negative UTC-offset browser timezones.
• Time values display without an invented date or timezone conversion.
• Duration values render and sort/chart numerically with documented precision and unit.
• The frontend never invokes the datetime renderer for Date, Time, or Duration.
• Malformed/unsupported temporal values degrade safely instead of rendering Invalid Date/NaN.
• Backend and generated frontend types match the actual payload fields.
• Cross-backend contract tests cover min, max, mean, std, quantiles, histogram boundaries where applicable, missing values, and empty/all-null columns.
• Frontend tests run under at least one negative-offset timezone (for example America/Los_Angeles) and verify Date behavior.
• Upgrade/schema compatibility is documented, including how older SDK payloads are rendered.

Suggested implementation areas

• pandas routing and serialization:
  • ui/sdk/src/hamilton_sdk/tracking/pandas_stats.py
  • ui/sdk/src/hamilton_sdk/tracking/pandas_col_stats.py
• Polars routing and serialization:
  • ui/sdk/src/hamilton_sdk/tracking/polars_stats.py
  • ui/sdk/src/hamilton_sdk/tracking/polars_col_stats.py
• Observability schema/models and generated API types
• UI grouping and renderers:
  • ui/frontend/src/components/dashboard/Runs/Task/result-summaries/DAGWorksDescribe.tsx
• Tests:
  • ui/sdk/tests/tracking/test_pandas_stats.py
  • ui/sdk/tests/tracking/test_pandas_col_stats.py
  • ui/sdk/tests/tracking/test_polars_stats.py
  • ui/sdk/tests/tracking/test_polars_col_stats.py
  • focused frontend temporal renderer/contract tests

Incremental path

Until this contract and its renderers exist, the safe schema-v0.0.3 fallback is for only true Datetime columns to use the datetime path and for unsupported Date/Time/Duration values to use unhandled. PR #1675 implements that fallback for Polars Time/Duration and should not be expanded into this larger schema/UI project.