`older_than` reads a naive datetime as UTC and expires snapshots it should keep

Open Beginner friendly
  • Is something wrong?
#4,182 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Rahul-cs50 is already working on this.

Since Oct 31, 2026.

  • #4220 by @Rahul-cs50 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
databases

Research direction

Start at older_than and follow how it converts the cutoff with datetime_to_millis; the issue’s reproducer shows the incorrect result for a naive local datetime. Check the existing expiration tests and add a case using a non-UTC timezone alongside the supplied reproducer. Done when naive local cutoffs preserve the expected snapshots east and west of UTC, while aware datetimes continue to work.

Written by the indexing model from the issue text.

Description

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Currently, older_than converts its cutoff with datetime_to_millis, which takes a naive datetime as UTC, so in Tokyo the docs' older_than(datetime.now() - timedelta(days=3)) keeps only 63 hours of snapshots. An aware datetime is read correctly.

import os, time, tempfile
os.environ["TZ"] = "Asia/Tokyo"  # UTC+9, any zone east of UTC does it
time.tzset()

from datetime import datetime, timedelta
import pyarrow as pa
from pyiceberg.catalog.sql import SqlCatalog

cat = SqlCatalog("c", uri="sqlite:///:memory:", warehouse=f"file://{tempfile.mkdtemp()}")
cat.create_namespace("ns")
t = cat.create_table("ns.t", schema=pa.schema([("x", pa.int64())]))
for x in (1, 2, 3):
    t.append(pa.table({"x": [x]}))

t.maintenance.expire_snapshots().older_than(datetime.now() - timedelta(hours=1)).commit()
print(len(cat.load_table("ns.t").snapshots()))
# 1, should be 3
# all three snapshots are seconds old, so none is older than an hour

West of UTC the cutoff lands early instead, so it keeps snapshots it should expire.

tested on pyiceberg 0.12.0 and main

Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Dominant language
Python
Stars
1.2k
Forks
664
Avg merge
2d 10h
Merged PRs (30d)
131

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.