[VL] RANGE window frames with interval or non-integral bounds fail the query instead of falling back

Open
#13,201 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
clickhouse, scala, spark
Domain
backend, databases

Research direction

Start by reading PullOutPreProject and WindowFunctionNode.setBound, then compare the ClickHouse supportWindowExec checks with the Velox and Bolt handling described in the issue. Reproduce the date-key interval and decimal-key bound cases where the relevant backend is available. Done means these cases fall back to vanilla Spark instead of failing; the issue does not name specific test files.

Written by the indexing model from the issue text.

Description

Backend

Velox (the window parts also apply to Bolt and ClickHouse, see below).

Bug description

Two kinds of RANGE window frames fail the query instead of falling back to vanilla Spark.

1. A date order key with an interval bound fails planning on Velox and Bolt. PullOutPreProject pre-computes RANGE frame boundaries for date keys by turning the key into an integer (datediff from epoch) and adding the bound to it. That only works for an integer bound. With an interval bound it builds Add(int, interval), which never resolves, and planning fails:

select l_commitdate,
       count(*) over (order by l_commitdate range between interval '1' month preceding and current row)
from lineitem
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object

range between 1 preceding and current row on the same date key works.

2. A non-integral bound fails the query with NumberFormatException on ClickHouse. WindowFunctionNode.setBound reads a literal frame bound with Long.valueOf(bound.eval(null).toString()). Spark casts a RANGE bound to the order key type, so on a decimal key both 1.5 PRECEDING and 10 PRECEDING arrive as -1.50 / -10.00. The resulting NumberFormatException is not one of the exceptions validation turns into a fallback, so the query fails. ClickHouse's supportWindowExec does not check frame bounds, so a query like this reaches it:

select order_id, count(*) over (partition by category order by price
                                range between 10 preceding and 10 following)
from orders  -- price DECIMAL(18,2)

Velox and Bolt already reject non-integral sort keys before this point. The ClickHouse path was traced by reading the code; I could not run the ClickHouse backend.

Gluten version

main (2681cc9a3)

Spark version

Spark 3.5

This issue was written with the assistance of AI (Claude Opus).

Dominant language
Scala
Stars
1.6k
Forks
679
Avg merge
4d 16h
Merged PRs (30d)
84

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/gluten

All issues in apache/gluten

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.