Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Visitor and evaluator edge cases can over-prune files or mishandle nulls

Open
#3,498 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
data, testing-qa

Research direction

Start with _StrictMetricsEvaluator.visit_not_equal, visit_not_in, and eval, then inspect the comparison methods and visit_not_nan on ResidualVisitor. Run the existing tests for stats, nullable partitions, and NotNaN behavior; done means the edge cases have consistent semantics without unsafe pruning or null-handling errors.

Written by the indexing model from the issue text.

Description

Several visitor/evaluator edge cases appear unsafe or inconsistent:

  1. _StrictMetricsEvaluator.visit_not_equal / visit_not_in return ROWS_MUST_MATCH when a file can contain nulls or NaNs. Example stats with [null, 5] or [NaN, 5.0] and lower/upper bounds both 5 return true for NotEqualTo("x", 5) / NotIn("x", {5}), even though one row does not match. This can incorrectly mark whole files deleted.

  2. _StrictMetricsEvaluator.eval returns ROWS_MUST_MATCH for record_count <= 0. record_count=0 is vacuously true, but record_count=-1 is unknown per the local comment; even AlwaysFalse() returns true.

  3. ResidualVisitor comparison methods directly compare partition values to literals. A nullable identity partition value of None with LessThan("x", 1) raises TypeError, while row evaluation returns false.

  4. ResidualVisitor.visit_not_nan(None) returns AlwaysFalse, while expression evaluation treats NotNaN(None) as true. Existing tests encode both behaviors, so the semantics are inconsistent.

Validated against the current tree; examples use stats/partition shapes already supported by the repo tests.

Dominant language
Python
Stars
1.1k
Forks
589
Avg merge
2d 2h
Merged PRs (30d)
70

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.