Spark 4.0 / DBR 14.2+ - bleeding edge changes

Open

#787 opened on Jan 26, 2024 11 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 25/100

Issue type: Bug
Clarity: Needs clarification
Activity status: Stale
Tech stack: scala, spark
Domain: data

Research direction

Start by reviewing SPARK-44913 and the related discussions in #755 and #300 to understand the StaticInvoke changes and prior compatibility approach. The issue needs a settled route between a bleeding-edge release and a separate swappable compatibility jar before implementation can begin.

Written by the indexing model from the issue text.

Description

enhancement feature

Per my comment on #755 DBR 14.2, 4.0 and likely all later versions includes SPARK-44913 StaticInvoke changes.

Whilst this hasn't yet been backported to 3.5 branch it could well end up there.

I'm happy to fork and publish a bleeding edge / non-standard frameless if needed but I also wonder if a compat layer as a separate swappable jar is the best route similar to #300 for example.

What is the collective preferred route to fixing / working around this?

Dominant language: Scala
Stars: 895
Forks: 135
Avg merge: 1d 16h
Merged PRs (30d): 3

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

RS_PixelAsPoint raises out of grid where RS_PixelAsCentroid and RS_PixelAsPolygon extrapolate Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

apache/sedona#3395 · Opened Sep 23, 2026
Add full support for Excel and XML exports as email attachment Open

Area: Excel support

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

orbeon/orbeon-forms#7893 · Opened Sep 23, 2026
bin/verify-exercises-in-docker can never pass: lens-person's example needs scalaz, which the test runner does not ship Open

x:action/fix x:module/practice-exercise x:size/small

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

exercism/scala#891 · Opened Sep 23, 2026
incorrect message on bad internet connection Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

lichess-org/lila#21788 · 2 comments · Opened Sep 23, 2026
docs: fix stale and missing native write details in iceberg-writes.md and iceberg.md Open

area:Iceberg area:writer documentation requires-triage

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

apache/datafusion-comet#6147 · Opened Sep 23, 2026

More Scala issues